This is the fifth of eight stories in our Web feature on self-experimenters.

When Deb Roy and his wife have overnight guests that might encounter their two-and-a-half-year-old son—the couple is withholding his name to protect his privacy—the first thing they do is ask their visitors to fill out a consent form. Unusual, for sure, but the couple is merely trying to make people aware that their actions and voices may be captured by the 11 fish-eye cameras and 14 microphones hidden around their Cambridge, Mass., home listening in on nearly every sound their son has ever uttered.

The two main goals of the setup: to understand how children acquire language and use the intelligence gleaned to teach robots to talk.

Roy, 39, head of the cognitive machines group at the Massachusetts Institute of Technology's Media Lab, is documenting every parent–child "conversation" in what he calls the Human Speechome Project. He estimates that by the time he finishes the recording phase of the project later this year, he will have collected an estimated 200,000 hours of video and multitrack audio data—or about 70 percent of the child's first two years of waking life, along with part of year three.

Roy initiated the project after hitting a dead-end in his robotics research on a particular bot named Toco. As a graduate student at M.I.T.'s Media Lab, he wanted to teach a robot to talk, so he programmed Toco with sophisticated image and speech processing software combined with machine-learning algorithms that he hoped would do the trick. But when Roy placed a ball in front of Toco's camera, he realized the machine could not fathom the difference between the meaning of "ball" (the object) and "round" (the object's property), both of which were represented the same way in its computer memory.

He decided that the first step to overcoming this problem was figuring out how young children learn to solve the same problem.

Researchers have tried before to unravel the process of language acquisition, usually by eavesdropping on mothers and their children for a few hours in a laboratory or in a home setting. But it was difficult to draw any hard-and-fast conclusions from such limited observations. "In developmental psychology there has long been a trade-off between gathering lots of data from a small number of children or a small amount of data from a much larger number of children," says Harvard University linguist Steven Pinker, who proposed a similar idea several years back. In cases—like Roy's—"the findings have held up well when extended to larger samples, as long as one looks at basic questions in which normal children are unlikely to show huge individual variation."

Roy, an engineer by training, plans to put his enormous data set through the wringer in an attempt to identify patterns in the way he, his wife (Rupal Patel, a professor of speech language pathology at Northeastern University in Boston) and the young one speak as common concepts are taught and lessons are learned. (He notes that when both of your parents are professors, "you probably have to listen to a lot of talk.") He hopes to reconstruct the mental processes that enable a child to gradually attach meaning and expression to words by studying certain clues: which word was spoken (e.g., "food?"); who said it (Patel); what the speaker wanted at the time (to get the kid into his high chair); and the tyke's response ("no").

1 Comments

1. 1. Ramesh1 07:55 AM 3/28/08

   Creating Robot just like man, What scientists actually aim?Is Robot have consciousness, is he have emotion, can he weep when his mother die,can he joyful and selfsatisfied when he achieved his aim?Is this dummy benifit to mankind or harmful. If he have no consciousness, artifical man is most dangerious to mankind than atombomb.What scientists want to achieve creating Robot?

   Reply | Report Abuse | Link to this