Huffington Post uses A/B testing on its headlines

Posted by Nestor Bailly on October 15, 2009 at 12:00 PM

A/B testing is a standard of user research that randomly presents one half of your audience with one version of content and the other half with a different version to compare the performance results. The version with a higher rating, be it more clicks or views or qualitative performance, then gets chosen as the only version presented.

The Huffington Post applies this method to its headlines, Nieman Labs reports. A story is given two headlines, and the one that gets more clicks in a five-minute period (apparently enough time for a high traffic site) is selected as the permanent headline.

Given the amount of care that is placed in choosing traditional newspaper headlines and magazine covers, this type of testing does not come as a surprise especially given Huffpost's recent expansion and development.

Huffpost developed their own A/B testing system and has not revealed any large trends, besides that placing the author's name above a headline generates more clicks than not including it with the headline.

Although A/B testing has large benefits as a testing method, it can suffer from its limited scope and one-time applicability. By focusing on one item only, it ameliorates short-term change and adjustment, at the potential cost of long-term quality focus.

But for headlines it is a perfect tool. Amazon was a pioneer in its use, and Google uses it massively to test even subtle changes. In fact one of their chief designers left over excessive use of A/B testing and spoke out against it.

Newspapers can and have used an adapted version of the test, by publishing one headline or content in one region and a different version elsewhere, for example.

Source: Nieman Journalism Labs