By Sharon Weinberger of Nature magazine
It is every government's dream: a system that can predict future events such as riots, political upheavals and the outbreak of wars. Last week, a collection of academics and private businesses was scrambling to meet the deadline for proposals for research aiming to do just that.
The Intelligence Advanced Research Projects Activity (IARPA), a research arm of the US intelligence community, is sponsoring the work under the Open Source Indicators (OSI) program. The three-year project, with an unspecified budget, is designed to gather digital data from a range of sources, from traffic webcams to television to Twitter. The goal, according to IARPA, is to provide the intelligence community with predictions of social and political events that can "beat the news".
Initially, the OSI project will focus on Latin America, which has abundant publicly available data and offers a convenient test bed for researchers' models. Those models will build on strategies that have already shown promise for predicting disease outbreaks and consumer behavior, and which are becoming increasingly popular with US national security agencies (see Nature 471, 566-568; 2011).
Indeed, the OSI project is one of many being sponsored by the US national security community, which seeks to meld mathematics, computer science and economics with the social sciences, creating a new field of social and political forecasting that has often been compared to Isaac Asimov's concept of 'psychohistory'.
At the Center for Collective Intelligence at the Massachusetts Institute of Technology in Cambridge, computer scientist Peter Gloor has been working with colleagues to build models that can predict consumer behavior, such as ticket sales for Hollywood films, using a range of online sources including social media. "We're up to 90% accuracy" for predicting box-office returns, says Gloor, who is part of a team applying for OSI funding.
John Brownstein, an epidemiologist at Harvard Medical School in Boston, Massachusetts, is working with a group that analyses international news sources, government data and social media to provide an early warning of disease outbreaks. He is also applying to work on the IARPA project. "In many cases, what we are searching for are patterns of activity that would not only apply to disease events but to conflicts, environmental disasters and other forms of social disruption," he says.
However, there is disagreement about how effective such biosurveillance programs have been. Picking up hints of an H1N1 outbreak is less useful if it is swamped by false alarms, says James Wilson, a medical doctor and the co-founder of Ascel Bio in New York. "There is a huge difference between a computer harvesting data and beginning to move communications so that people begin to contemplate action," says Wilson, whose company is applying for OSI funding.
Data crunching
The approach is nevertheless catching on in the financial sector. Christopher Ahlberg, the chief executive of Recorded Future in Boston, also applying to the OSI project, says his company has proprietary software that can crunch through 300,000 sources an hour for clues to future stock movements. The company has already received investment from In-Q-Tel, a venture-capital firm in Arlington, Virginia, founded by the US Central Intelligence Agency, and is mining online data sources, such as Twitter, in an effort to predict cyber attacks and developments in the ongoing 'Occupy Wall Street' protests.
There is already plenty of published literature on predicting stock market volatility based on open sources, says Kalev Leetaru, a computer scientist at the University of Illinois at Urbana-Champaign. But that's a long way from being able to predict a riot next Wednesday. Leetaru, who is also part of a team applying to be part of the OSI project, says he asked IARPA whether it would consider a broader approach that involved predicting trends, such as the Arab Spring movement, rather than focusing on specific events (see Nature; 2011). The answer, he says, was "no".
"In my mind," Leetaru says,"this would be akin to the National Institutes of Health announcing it was terminating all funding for work on cancer drugs that targeted individual cancers or that slowed cancer down, and instead announcing that they would only fund a single miracle pill that you take and your cancer is cured the next morning."
Robert Albro, an anthropologist at American University in Washington DC, and an expert in Latin American social movements, believes that IARPA is mistakenly presuming that social media will provide high-quality data. "Just because data are available doesn't make them good, or the key to the kingdom," he says.
A bigger question is whether models used to measure consumer preferences and disease outbreaks are applicable to the complex world of social change and political events. Albro says that such models make faulty assumptions about what motivates humans, and he worries "that companies concerned about consumer behavior are now driving how IARPA thinks. That's a leap of faith.
This article is reproduced with permission from the magazine Nature. The article was first published on October 17, 2011.
See what we're tweeting about
4 Comments
Add CommentAre there even the remotest of ethical concerns here, or is the scientific establishment just one more mindless resource for big brother?
Reply | Report Abuse | Link to thisGive me a break.......
Remember the Mothers of Invention's "Who Are the Brain Police"? I guess we now know.....
Reply | Report Abuse | Link to thisI predict that the civil war will hit Jan 1 2013+ - 1 week
Reply | Report Abuse | Link to thisLet us set aside the question of whether spying on communications between citizens and their friends agrees with the 4th and 5th Amendments of the Constitution of the United States. Any literate person can see that such spying is illegal.
Reply | Report Abuse | Link to thisRather let us turn to the practical side the of question, assuming for hypothetical purposes that such spying (also known as warrantless wiretapping) is legal. If you want to know a problem and predict unrest amongst the citizenry then I'm quite sure that the citizens themselves will be eager to offer their opinions on the matter.
Might I suggest a more cost-effective way of reducing unrest that does not involve skulduggery?
How about solving some social problems? If millions of people cannot find jobs to pay their bills and pursue their goals in life then the policy of any sensible government would be to ensure they have jobs and money. How on Earth is spying on the citizenry going to move us towards that worthy objective?
I see that perhaps there is a collection of vastly better informed and more sophisticated experts than this poor amateur that have somehow got it worked out that spying on citizens and infuriating them shall somehow lead us all to a special utopia filled with harmony and good will... if only we can cure the bad thinking that accompanies pangs of hunger without actually putting food on table.
This does, indeed, rival any panacea with which I am familiar and amounts to nothing short of a miracle. Such leaps of faith do not come easily to me, one among many a poor soul burdened with life's experiences both pleasant and otherwise. So, I believe they owe us some sort of explanation as to why the government is spending money on this spying technology instead of working to resolve the ills with which they are so apparently concerned.