Yahoo to cover news based on search data

Posted by Colin Heilbut on July 5, 2010 at 2:17 PM

Yahoo has announced that it plans to start using data gleaned from the millions of search terms it processes every day in order to select news stories to cover. Starting Tuesday, Yahoo! will introduce a news blog that will rely on search queries to help guide its reporting and writing on national affairs, politics and the media. Yahoo software continuously tracks common words, phrases and topics that are popular among users across its vast online network. A team of people will analyze those patterns and pass along their findings to Yahoo's news staff of two editors and six bloggers who will use the information to help write the new blog, called The Upshot which will go live tomorrow at news.yahoo.com/upshot.

In an interview with the NYT, James Pitaro, vice president of Yahoo Media, said that "We feel like the differentiator here; what separates us from a lot of our competitors is our ability to aggregate all this data. This idea of creating content in response to audience insight and audience needs is one component of the strategy, but it's a big component."

As an example of the advantages of search technology as a editor, Mr. Pitaro often points to one of the most popular articles to appear on Yahoo's sports news site during the 2008 Summer Olympics.

Yahoo! had been monitoring search traffic patterns and noticed that its users kept trying to find out why divers would shower after they got out of the water. So Yahoo sports writers looked into the question and posted an item titled "The mystery of the showering divers." (It turns out the warm water from the showers keeps divers' muscles limber. Their muscles contract when they emerge from the warm water into the cool air.)

"So while our competition was covering a lot of the bigger, broader topics, we were covering topics that were a little bit more behind the scenes," Mr. Pitaro said.

The prospect of abandoning the traditional editorial process is not without its challenges. The first problem is that search-terms do not necessarily reflect current events. Among the 'trending' terms at the moment, according to Google, are hot dog (perhaps, though not necessarily, in reference to the recent Takeru Kobayashi incident?), the new twilight movie and the 1812 overture.

Beyond the technical challenges, the The Yahoo model also flies in the face of a centuries-old approach to disseminating the new which is likely to be frowned upon by journalism purists. "There's obviously an embedded negative view toward using any type of outside information to influence coverage," said Robertson Barrett, chief strategy officer of Perfect Market Inc., a company that helps news organizations make their content more detectable to search engine algorithms.

The Upshot's new editor, Andrew Golis, says that his blog would "use the search data as a supplemental tool... It's an asset. It's a totally amazing and useful tool that we have at Yahoo. But it does not lead Yahoo editorial content". The ability to combine traditional journalism with Yahoo's proprietary feed of real-time search data may make The Upshot an very interesting blog to watch.

Sources: NYT