This paper was written by Meila (MIT) and Heckerman (Microsoft Research) and published at the Machine Learning journal in 2001.
This tech report compares three batch clustering algorithms: EM, CEM, and HAC and also investigate the effect of three different initialization schemes on the final solution produced by the EM algorithm.
The clustering algorithms first use the log posterior probability of model structure given the training data log P(K|Dtrain) to select model structure, and then learn the parameters of a given model structure. The EM algorithm is iterative, and in the E step, assigns the case fractionally to each cluster. The CEM algorithm assigns the case fully to the class k that has the highest posterior probability given the document and the current parameter values. The HAC algorithm merge smaller clusters by recursively merging the two clusters that are closest together until only K clusters are left. The authors derive the distance metric for mixtures of independent multinomially distributed variables.
The three initialization methods experimented are: the random approach where the parameters are independent of the data, the noisy-marginal method (data dependent), and the method of using HAC on a random sample of the data. Several performance criteria are used. For the model structure, the log-marginal-likelihood criterion and the difference between the number of clusters in the model and the true number of clusters are used. For the entire model, one criterion is the cross entropy between the true joint distribution for X and the joint distribution given by the model, and another criterion is the classification accuracy. Running time is also used to compare algorithms.
Experimental results from two datasets were reported. The synthetic dataset was constructed using the MSNBC news service, and the digits dataset consists of images of handwritten digits made available by the US Postal Service Office for Advanced Technology. For both datasets, EM outperformed CEM for all initialization methods and for all criteria (except that CEM is four times faster than EM). After constraining EM run time to match CEM, EM still performed better. When comparing EM versus HAC (initializing using Marginal method), EM is clearly better.
When comparing initialization methods, for the Synthetic dataset, Marginal and HAC are always better than Random (for all criteria) with no significant differences between themselves, but Marginal is much faster than HAC. For the digit6 dataset, there is no clear winner.
Picture of the Day:
First time ever a rescue helicopter landing in Bryce Canyon, Utah. The chopper landed right on the top of the ridge. Got to give the pilot some shout out for his awesome skills!
A voice came from outside the iron-door, “Mr. Ren, it has been very hot in the last couple of days. May I ask if the revered mister still enjoys his good health?”
As soon as Linghu Chong heard the voice, he recognized it instantly. It was the voice of Mr. Black-White. If he had come to his cell one month before, Linghu Chong would have shouted all kinds of invective at him with no scruples. However, after many days of imprisonment, his anger had mollified a great deal and he was able to think calmly.
“Why did he call me Mr. Ren? Has he come to the wrong cell?” he thought to himself and decided to remain silent and listen on.
“Every two months, I come and ask the revered mister the same question. Today is the first day of July, so please allow me to ask once again: Will the revered mister grant my request?” Mr. Black-White continued, his tone respectful and cautious.
Linghu Chong laughed inwardly. “He got the prison cells all mixed up and must have thought I am the Revered Mr. Ren. How careless of him!” But immediately after, he felt a chill in his heart. “Mr. Black-White obviously is the most meticulous one among the four Masters of the Plum Manor. It might be possible for Mr. Bald-Brush or Mr. Paint to mix up the prison cells, but how could Mr. Black-White actually make such a mistake? There must be a reason behind this.” With that in mind, he kept his silence.
Mr. Black-White continued, “Revered Mr. Ren, valiant and capable all your life, why let yourself languish with the decaying dust in the underground dungeon? My words are as firm as the mountain. If the revered mister grants me my request, I promise I will help the revered mister get out of this prison.”
Linghu Chong’s heart thumped wildly. Many thoughts raced through his mind, but he failed to put them in order. What was Mr. Black-White really after and why did he speak this way? He simply couldn’t figure it out. Then he heard Mr. Black-White asking a second time.
“Will the revered mister grant my request?”
Linghu Chong knew that this could be the very opportunity for him to escape. Despite any possible ill intentions, it would still have been a much better alternative than getting stranded forever in the underground prison for some unknown reasons. But having no way of identifying Mr. Black-White’s true intention, he bit his lip and remained silent, afraid that he might say something wrong to ruin this precious opportunity.
“Revered Mr. Ren, why aren’t you speaking up?” Mr. Black-White heaved a sigh. “Last time, when that boy Feng challenged you in the art of sword, you didn’t mention a single word about my request in front of my three sworn-brothers. I am truly grateful for the kindness. I thought to myself, the sword fight must have brought back the many fond memories and the lofty sentiments in Revered Mr. Ren’s heart, and reminded him of the vast world outside. If your Excellency could break out of the dark dungeon, the entire world would be yours for the taking, and you can snuff out any mundane life at will, whether male or female, old or young; and no one would ever dare defy your wish. Wouldn’t that be exhilarating? Granting the request would not have brought any nuisance to your own. Why wouldn’t you ever consent in the past twelve years?”
Linghu Chong could sense the sincerity in his voice. Apparently he really thought he was addressing senior master Ren, which puzzled Linghu Chong even more. He listened on as Mr. Black-White begged again and again for him to agree to the request. Linghu Chong really wanted to know more details about the request, but he reckoned that the situation would take an immediate bad turn if he opened his mouth. Holding his tongue forcibly with great effort, he dared not make the slightest sound.
“If the revered mister is still so stubborn, I guess I’ll visit him again in two months,” Mr. Black-White concluded. He suddenly let out a few chuckles. “The revered mister didn’t scold me this time. Maybe there’s a favorable turn in the situation. I hope the revered mister will give my request some good consideration in the next two months.” After those words, he turned around to exit.
Linghu Chong became quite worried. It would be another two months before Mr. Black-White would return, and how could he stand another two months of the miserable life in the pitch-black prison? He waited until Mr. Black-White had taken several steps and then spoke up in an intentionally lowered and coarse voice.
“What request shall I grant?”
Mr. Black-White turned around at once, and with a swift leap, he had returned back to the square-shaped opening.
“You will…you will grant me my request?” His voice trembled.
Linghu Chong turned to face the wall and covered his mouth with his hand.
“What request?” he said in a muffled voice.
“In the past twelve years, I would take on great risks to come here six times each year, just so that I could implore for the revered mister to consent. Why is the revered mister asking when the answer is so obvious?”
“I forgot!” Linghu Chong snorted.
“I would like the revered mister to teach me the secret of the magical art. Once I master the skills, I promise to let the revered mister out of this place.”
“Has he really mistaken me for that senior master Ren, or is this some kind of intrigue?” Linghu Chong pondered. Still having no clue about Mr. Black-White’s true intentions, he mumbled something indistinct, not even knowing what he said himself.
Naturally, Mr. Black-White didn’t understand a word of his, and had to ask again and again, “Will the revered mister grant it? Will the revered mister grant it?”
“You don’t live up to your promise. You won’t fool me,” Linghu Chong replied.
“What kind of guarantee would I have to provide so the revered mister will believe my words?”
“What do you have in mind?”
“I understand the revered mister’s worries. What if I break my promise after the revered mister teaches me the secret of the magical art and still not let him out? There is no need to worry! I have it all arranged, and the revered mister can put his trust in my arrangement,” Mr. Black-White answered.
“What arrangement?”
“Does this mean the revered mister will grant me my request?” Mr. Black-White asked quickly, his voice filled with excitement.
Linghu Chong thought very quickly, “He wants me to teach him the secret of the magical art. Where the heck would I find this secret of the magical art? But I might as well check out his arrangement. If he really will let me out of this place, I could just give him those meditation techniques carved on the steel plate. Who cares if it works or not, it doesn’t hurt to give him some false hope.”
When Mr. Black-White didn’t hear an answer, he added, “After the revered mister teaches me the magical art, I would have automatically become an apprentice of the revered mister. In our cult, an apprentice who deceives his master is to be skinned and dismembered alive. In the past several hundred years, not a single one had managed to get away from the harsh punishment. How would I ever dare to not let the revered mister out?”
“I see,” Linghu Chong let out another snort. “Come back in three days to get your answer.”
“Why won’t the revered mister grant my request today? Why wait another three days in this dark dungeon?” Mr. Black-White pleaded.
Linghu Chong thought to himself, “He is even more impatient than I am. It might be a good idea to wait three days and see if he’s going to pull some clever tricks.” At that thought, he produced a loud, disgruntled humph as if he was very annoyed.
“Sure! Sure! I shall return in three days so I could ask advice from the revered mister again!” Mr. Black-White hurriedly agreed.
Linghu Chong listened as Mr. Black-White walked out of the underground tunnel and closed the iron gate while many unsettling thoughts swirled inside his head.
“Could he really have mistaken me as the Ren-named senior master? He is a very meticulous man. How could he make such a silly mistake?”
Suddenly, an idea struck him. “Could it be possible that Mr. Huang-Zhong found out about Mr. Black-White’s visits and secretly moved senior master Ren to another cell, then put me in this cell instead? That’s got to be it! In the past twelve years, Mr. Black-White had been paying a visit every other month. It’s very likely that someone found out about it. It must have been Mr. Huang-Zhong who set up this scheme clandestinely.”
Then, he remembered what Mr. Black-White had said earlier. “In our cult, an apprentice who deceives his master is to be skinned and dismembered alive. In the past several hundred years, not a single one had managed to get away from the harsh punishment.”
“Our cult? Which cult?” he thought it over. “Could it be the Demon’s Cult? Is it possible that the senior master Ren and the Four Skunks of Jiangnan are all members of the Demon’s Cult? What kind of game are they playing? Why are they dragging me into this mess?”
As soon as the name “Demon’s Cult” came to his mind, he found the entire matter surreptitious and beyond his comprehension, and decided to not ponder upon it. Instead, he concentrated only on two things: “Was Mr. Black-White sincere or was he just acting it up? How shall I answer him when he comes back in three days?”
All kinds of strange idea went through his mind, but no matter how hard he tried, he still couldn’t figure out Mr. Black-White’s true intentions. After some time, he became extremely weary and fell asleep. By the time he woke up, his first thought was, “If brother Xiang were here, with his experience and knowledge, he would have guessed Mr. Black-White’s intention in no time. That Ren-named senior master was also very bright, likely even smarter than brother Xiang…oops!”
He suddenly cried out and jumped to his feet. After the nap, his brain functioned much better and he realized something.
“For the last twelve years, senior master Ren never granted him the request. Why? Because he knew very well what would happen if he did. As sharp and experienced as he is, he clearly knew the pros and cons of the matter.” Then he thought to himself, “Although senior master Ren should not grant the request, I am not senior master Ren. Why can’t I?”
He knew he shouldn’t do it. He could even smell the great danger involved, yet the hope of escape weighed heavily on his mind, and he would risk the greatest calamity just for an opportunity to break out from the dark underground dungeon. He soon made up his mind.
“When Mr. Black-White comes back in three days, I will consent to his request and teach him the inner energy cultivation and meditation techniques carved on the steel plate. Then I’ll just act according to his reactions.”
Now support the translator Lanny by following my blog and leaving comments! :)
Video of the Day:
One of the best fighting scenes in Kung Fu movies --- Jet Li's Fist of Legend
A few words first: To make it easier to find posts that interest you, I've added a search box to my blog (you probably have noticed it right above my post) that searches through all my blog posts but not anything outside of my blog. Just on the right side oft each post, you can also click on different blog labels to read my posts by category. At the end of each blog post, I've also included icons you can click to share the post with your friends using your favorite social network tools. Spread the good word if there's something you really enjoy! Okay, the real post starts below. ================================================================
Robot arm stabbing a human volunteer with a knife (Photo credit: IEEE Spectrum)
At the ICRA 2010 conference (IEEE International Conference on Robotics and Automation) that is currently ongoing in Anchorage, Alaska, some German researchers presented their latest research on the biomechanics of soft-tissue injury caused by a knife-wielding robot. The paper is titled "Soft-tissue Injury in Robotics." In other words, they wanted to find out what will happen if a robot holding a sharp knife erroneously stabs a person.And, no, I am not joking. The robot arm in the picture on the right is really holding a knife, and it really stabbed the guy's arm with it.
These researchers are from the Institute of Robotics and Mechatronics, part of DLR, the German aerospace agency, in Wessling, Germany, and they share the same dream with me --- that one day robots will be smart enough to take over kitchen duties and free us from the laborious duty of cooking. This task of course requires the robot to be able to handle a knife appropriately, so it can cut, chop, slide or dice during the course of preparing a meal. But what if it accidentally struck a human? With that question in mind, these researchers performed a series of experiment to investigate the severity of possible injuries and also designed a collision-detection to minimize the damage.
Various knives used in the experiments (Photo credit: IEEE Spectrum)
They mounted various sharp things, from knife to scissors to screwdriver (why does this somehow remind me of GTA San Andreas? Shudder!), to a DLR Lightweight Robot III, or LWRIII, a 7 degrees-of-freedom robot arm, and then tested the striking on a block of silicone, a pig's leg, and eventually, on the bare arm of a human volunteer. The collision-detection system turned out to be very successful, because the volunteer still has his arm.
The video below shows how the experiments were performed and how the robot arm performed differently with and without the collision-detection system (the real excitement is at the end of video). As a researching in Human-Robot Interaction myself, I couldn't help but imagine this poster in my head that reads, "Volunteer needed for a user study: Get Paid to be Stabbed by a Robot!"
But I am a little bit confused. Once turning on the collision-detection system, the robot will stop cutting/stabbing the human. The human is safe now, and so is that piece of steak! Three hours later, I'd be shouting in starved voice, "Where's my steak dinner?"
Note that the idea of a robot holding a knife would never be allowed in US universities. It would never get approval from the IRB (Institutional Review Board). See, we do things very differently here in the US, instead of knives, we give robots machine guns and missiles!! And there will be no danger to US citizens, because we send these robots to other countries! LOL!
MQ-9 Reaper Predator UAV
SWORD Robot
I think I am a bit off topic now, so let's get back to these German researchers. If I remember correctly, I actually saw a video from HRI 2007 made by the same guy demonstrating how he would let a powerful robot arm punch him in the head (Sorry I am having a hard time finding this video now). The robot arm would VERY QUICKLY slow down when it detects the collision, thus sparing the guy's life. Well, hats off to the guy!! Comparing to him, I am a coward, because I would never put myself under such conditions --- because I am a terrible programmer, and I have lots of bugs in my code. And my admiration for him went sky-high when I realized they also performed the following experiments. Ouch!!
Anyway, I think it will be a long time before we actually have knife-wielding robots that roam our homes. When I program my robots, I actually intentionally make it not touch things such as knives, gas stoves, and explosives. But I bet you this day will eventually come, and a lot of lawyers are going to get rich.
Drinking excessive amount of Mountain Dew and staying up till 4am can lead to severe stomach cramping and internal bleeding and many days of lost productivity.
If you've ever made purchases online, or signed up for any kind of account on the Internet (forums, subscriptions, etc.), or installed software applications on your computer, then you must have seen screens where you have to agree to some kind of terms and conditions, or EULA (standing for End User License Agreement). How many times, however, do you actually read through those pages after pages of dry, dull, impossible to understand legal languages, especially on a computer screen?
Well, if quickly clicking through things has grown into a habit for you, it's probably time to reconsider it now.
According to this article from Newslite and this article from geeks.com, GameStation, on UK online games retailer, added the "immortal soul clause" to its online purchase agreement basically saying the customers would grant them the right to claim their souls. More than 7,500 customers (88%) agreed to the clause without really reading the agreement details, and now GameStation legally owns thousands of souls. Here below are the exact wording of the sneaky clause:
By placing an order via this Web site on the first day of the fourth month of the year 2010 Anno Domini, you agree to grant Us a non transferable option to claim, for now and for ever more, your immortal soul. Should We wish to exercise this option, you agree to surrender your immortal soul, and any claim you may have on it, within 5 (five) working days of receiving written notification from gamesation.co.uk or one of its duly authorised minions. We reserve the right to serve such notice in 6 (six) foot high letters of fire, however we can accept no liability for any loss or damage caused by such an act.
If you a) do not believe you have an immortal soul, b) have already given it to another party, or c) do not wish to grant Us such a license, please click the link below to nullify this sub-clause and proceed with your transaction.
Luckily, 12% of the customers did notice a small tick box at the bottom of agreement allowing them to opt out of the clause, and earned themselves a £5 gift voucher by doing so. But really, how many of the 12% customers actually read the terms and services agreement? My guess would be very few.
Although this probably was part of an April Fool's Day joke (the vouchers were no joke), and GameStation has no intention to ever claim their rights on the souls (in fact, they emailed all customers nullifying the clause), this clearly shows the general public opinion on these agreements. Most people think that if most people don't read these things, they are no longer valid. However, the truth is that they will most likely be legally binding (not the soul clause, of course) unless challenged in a court of law. And do you ever want to get into a legal battle with large companies who employ an army of lawyers?
There are, actually, some people who take these agreements very "seriously!" Anne Loucks from obesso.com created a clever way to get herself out of legal troubles with EULAs. Using a piece of cardboard that reads "Kitty Agrees" with a protruding piece underneath, she would have her cat to step on the board, which pushes the spacebar underneath and agrees to an EULA for her. So worst comes to worst, it would be the cat losing its soul, and not Anne!
Putting aside the legality of these actions of both the businesses and users, clearly the system is not working very well. I know for regular business contracts or terms and agreements, critical elements are required by law to really stand out (bold, larger size, etc.). I am sure a better standard and approach will have to be created soon, because soon or later, somebody is getting sued over this.
Picture of the Day:
Brian Ghidinelli's creative reuse of his old equipments.
This paper is written by Steinback, Karpis, and Kumar, University of Minnesota, published at KDD workshop on text mining, 2000.
This paper presents the results of an experimental study of two main approaches to document clustering, agglomerative hierarchical clustering and K-means (standard K-means and bisecting K-means).
Example of Hierarchical Agglomerative Clustering (HAC)
The two basic approaches to generating a hierarchical clustering are agglomerative and divisive. The paper evaluated agglomerative techniques in the comparison. It then described the agglomerative clustering algorithm, the K-means algorithm, and the bisecting K-means algorithm in details.
Visualization of the K-means algorithm
Three evaluation metrics are used in the experiments, and they include two external quality measure, entropy, F measure, and one internal quality measure, overall similarity. The paper described each measure in detail.
Eight data sets were used in the experiments: 5 from TREC, 2 from Reuters-21578, and 1 from WebACE. Performances of three agglomerative hierarchical techniques, Intra-Cluster Similarity Technique (IST), Centroid Similarity Technique (CST), and UPGMA were compared using F-measure and entropy. UPGMA is the best performing hierarchical technique overall, therefore, its performance is compared against standard K-means and bisecting K-means. The performances of bisecting K-means with refinement and hierarchical with refinement are also included in the comparison. In the experiments, the authors used many runs of the regular K-means algorithm and also used incremental updating of centroids.
Experimental results show that the bisecting K-means technique is better than the standard K-means approach and as good or better than the hierarchical approaches when using the three evaluation metrics mentioned. Also the time complexity of bisecting K-means is linear, which makes it very attractive.
The authors argued that the agglomerative hierarchical clustering didn’t do well because nearest neighbors of documents often belong to different classes. K-means and bisecting K-means algorithms do better because they rely on a more global approach. They also believe that bisecting K-means does better than standard K-means because it produces relatively uniformly sized clusters.
Video of the Day:
The Honest $10000 SPAM
Even though miracle happens, still, don't click on suspicious links or give out your bank information. The Nigerian connection at the end of the video is simply hilarious!!
