Twice in the past week Jared Keller of TheAtlantic.com has posted stories of how scholars have found correlations between trends on Twitter and changes in the human condition (for better or worse). Academics have developed algorithms that can survey the ‘mood’ in the Twittersphere by seeking out specific terms or expressions of sentiment, then applying further ‘fuzzy logic’ algorithms to correlate the trends expressed in Twitter with such exchanges as stock markets and communicable diseases. What makes Twitter the sample base of choice for developing prediction models for such disparate experiences?
Johan Bollen of Indiana University-Bloomington and one of his grad students, Huina Mao, developed the mathematics that allowed them to cull public Twitter feeds looking for words meant to convey emotions. They developed the list of keywords from the generally-accepted standard of the ‘Profile of Mood States‘ drawn up by the MHS psychological assessment service. What they found especially interesting is that they could limit their search to those tweeting about the stock market and achieve a successful prediction rate of 73.3%, yet when they expanded the search to include terms that noted personal/emotional statements, their prediction rate jumped to 86.7%.
“Including this mood information leads to higher accuracy,” Bollen said. He stressed that their algorithm is highly simplified, and not the best stock market predictor anyone could come up with. But “we’re presuming on the basis of what we found, if you have some kind of super-duper algorithm and you add our time series, its accuracy will go up, as well.”
[When he and Mao added the emotional markers and the accuracy of prediction went up,]
“I sank into my chair. That’s a pretty big result,” Bollen said. “It was one of those ‘Eureka!’ moments.”
Quoted from his interview with Lisa Grossman at Wired.com.
My own ‘Eureka!’ moment was that people get paid hundreds of thousands of dollars claiming they have the business-school training and ‘scientific’ information about investing, when the best predictor is how the general public generally feel about life, the universe, and everything. Though the market price of the run-of-the-mill investment adviser or hedge-fund supervisor is not falling on this revelation.
If past results are not guarantors of future performance, then we should follow Bollen’s own advice: that more time and variables need to be given to the algorithmic model before we hand our portfolios over to it. But the idea of prediction via social media is being used in the medical arena as well.
Keller reports, “Between August 2009 and May 2010, associate professor for computer science Aron Culotta and two student assistants [at Southern Louisiana University] analyzed more than 500 million Twitter messages collected through the service’s application programming interface (API). By using a small number of keywords to track flu-related updates – ranging from headache to sore throat – Culotta’s team had a 95% success rate in matching the CDC‘s projected probability of flu outbreaks in the United States.”
What make Twitter streams such an attractive aggregate source? For one, they are publicly accessible and ‘followable’ (of course). The ease of quickly sharing one’s mood or physical well-being through a computer or cell phone suggests a data pool with a wonderfully wide demographic – though that demographic probably trends toward the younger. Moreover, the ubiquity of cell phones might mean that people ‘on the ground’ will post about their individual discomforts quickly, and from a bird’s-eye view we can witness the starting of an epidemic. Indeed, both sets of researchers stressed the fact that survey data might ultimately prove more accurate in tracing these phenomena, but such accuracy takes a great deal of time and effort for a marginally better percentage. The algorithms can work automatically and in real time:
Culota’s software is remarkably efficient. While the CDC produces weekly estimates for disease outbreaks, those reports typically lag a week or two behind due to the data collection process. Culota’s program is lightning fast and entirely automated, analyzing the bulk of each day’s Twitter messages and producing an estimate of the current proportion of people with the flu.
It will be interesting to see how the Groupthink of Twitter is used by others to predict World Series Winners (Texas Rangers in six, baby!) or midterm elections. As both of Mr. Keller’s posting note, Twitter has been considered a viable political barometer since 2008. Yet so too are these scientists aware that Tweeps can follow, search for, and comment upon erroneous information that might cause a spike of interest on a topic though there is no real-world equivalent. Mountaintop prophets and UFO searcher please take note…
