Man vs. Machine: Can Software Trump Editors in the Newsfeed Wars?

Xconomy National — 

Suddenly, digital newsfeeds are the hottest thing in consumer tech.

Big tech companies have been frantically opening their wallets in recent weeks to buy out small startups making mobile-friendly applications that use software to pull together news from innumerable sources around the Web.

Last week, it was Google paying a reported $30 million or more for Wavii, a Seattle-based startup that used language recognition and processing software to distill information from online news reports.

Two weeks earlier, professional networking website LinkedIn confirmed it was buying top-ranked newsfeed app Pulse for some $90 million.

A month ago, Yahoo got a flood of headlines for cutting its own $30 million check for Summly, another app that sliced news articles into a more mobile-friendly format—or, as Summly put it, “algorithmically generated summaries from hundreds of sources.”

This latest flurry of transactions follows CNN’s 2011 purchase of the app Zite, which pulls together news feeds for readers in a stylish, digital magazine-like layout.

What’s going on here? Think of it as a natural outgrowth of the social media age, which got a lot more people comfortable with ranking and rating the things they like, places they visit, and information sources they trust.

While Google, Yahoo, and LinkedIn haven’t been able to compete with Facebook’s ability to build a “social graph,” those companies are all clearly hoping that a user’s ability to personalize their own information and news feeds will give the providers deep insight into each consumer’s interests and preferences.

That kind of data, in an age when we’re all going to start generating more digital crumb trails with each passing year, will probably be a goldmine for advertisers—and the companies who can sell to them.

But it does set up an interesting question about quality. Right now, algorithm- and artificial intelligence-based apps that comb over disparate sources of information, pull out the most important parts, and reassemble them in a smaller package can do a good job of telling a user what’s going on in the broader news world.

So, for example, an algorithmic newsfeed might crawl the Web and see a lot of reports about the big college stars chosen in the first round of the NFL draft. It would spit out to the user a short, concise sentence: Kansas City Chiefs select Central Michigan offensive tackle Eric Fisher with the first pick in the draft.

When it works, it’s pretty cool. But in my experience, so far, the human-powered versions do it better—especially in critical situations.

Breaking News, a unit of NBC, employs a team of editors who pull together reports from a variety of sources into a “real-time feed that focuses on just what’s new.”

Breaking News, a unit of NBC, employs a team of editors who pull together reports from a variety of sources into a “real-time feed that focuses on just what’s new.”

Wavii is the example I’m most familiar with as a user. The ideas behind the app were very cool, and the technology they were employing was lauded as ahead of the curve and very smart.

The ambition was certainly aimed in the right direction—when it launched publicly about a year ago, CEO Adrian Aoun told me that “it makes you wonder why something like what we’re building doesn’t live inside Google Plus.”

But over several rounds of trying Wavii on my smartphone, I was never quite able to get the stuff I actually wanted to see consistently on my main feed. There was too much tech news and celebrity stuff for my reading interests, which are more general and local. The sources it linked to also were frequently some kind of off-brand website that was aggregating other news.

There is an up-front lag in these machine learning applications, of course—if you don’t use them, they can’t learn what you want. But I could never stick with Wavii long enough to get it tuned up better than my Twitter feed, which is where I get most of my news.

On the other side of this field are news services that take the old-school approach of hiring a bunch of people to read the news, sift through the headlines, and rank what they see as the top stories and most important developments.

Breaking News, a unit of NBC, is perhaps the most prominent of the new wave here. Based in Seattle, with offices in New York, London, and Portland, OR, it employs a team of editors who pull together reports from a wide variety of media sources into a “real-time feed that focuses on just what’s new.”

Another prominent real-people newsfeed startup is Circa, based in San Francisco. Circa also employs a team of editors who comb over the news of the day, extracting important facts and bits of information from longer, print- or Web-centric articles and regurgitating those points into a much more smartphone-friendly format.

There are plenty of others—Techmeme in the technology business news arena, which relies partly on editors to help compile a single page of big news, linking to the organizations that originally reported it. And there are some original-wave niche aggregator/bloggers, like Jim Romenesko in the media world—the leading place news people turn to for news about their industry.

This is much more expensive than an algorithm at any sort of large scale. The magic of software, both as a technology and a business, is that a digital machine can be built once and continue performing its task over and over. Even with constant updates in the software-as-a-service era, it’s much cheaper to crawl the entire Web with software than it would be to hire editors to do that task.

And we’ve had some recent examples of why a human decision is still needed. In the days following the Boston Marathon bombings, there were some long periods of information chaos as established news organizations and amateurs alike were spreading information that turned out to be wildly incorrect.

Breaking News general manager Cory Bergman summarized in this blog post how the staff decided to wait in one of these occasions, balancing a number of conflicting reports about whether there had been a suspect arrested early in the investigation. Despite reports from reputable organizations like CNN and The Associated Press, that news turned out to be incorrect—and Breaking News had done well by choosing to not amplify the incorrect reports, even though they were “out there.”

Now, an algorithm could probably be tuned to make some of those same calculations and come up with the same correct call. But the tools on offer today would be much more likely to bring you the latest from the most authoritative sources—even when their reports might look suspect to a practitioner, and eventually turn out wrong.

“Right now, computers can’t empathize,” Circa CEO Matt Galligan recently told AllThingsD. “We need to be able to understand a story, and right now, humans are the only ones that can truly understand a story, or the purpose, or the emotion.”

Of course, as a reporter, I have a very direct interest in the idea that humans are better news editors than algorithms. But I also don’t doubt the ability of software programs to drastically improve on their current abilities.

That’s definitely where Google’s heading. During an appearance at Harvard, in the same week that his company bought Wavii, Google chairman Eric Schmidt said a future of personally tuned newsfeeds was a big reason that the company shut down the Google Reader RSS application.

“The core presumption there is that people will have personalized or personally identified accounts where they’ve chosen to log in—Google Plus does this—and you’ll get your information through those mechanisms, as opposed to RSS feeds,” Schmidt said. “RSS was a tremendous technology from 15 years ago … there are just newer technologies coming along.”

“What I would suggest is that the real thing you want is not a feed, but rather an AI algorithm that knows roughly what you care about and is looking all the time to select the best and most interesting feeds,” he continued. “And we’re on the cusp of being able to do that technologically, based on your preferences and using machine intelligence and data mining. And if Google doesn’t do it, other people will.”