Robot crowd control. No, it's not its not the title of a new disaster movie starring Will Smith. It's CrowdControl, a start-up, lauched today, which aims to judge the accuracy of crowd-sourced information using artificial intelligence technology.

Programs to work out how people are feeling on social media already exist, but they are based on algorhythms that often don't work very well. Machines can identify key words, but aren't as good at detecting sentiments behind words. Apart from anything else, robots don't get sarcasm. Humans, who do a better job, can't analyse the same volume of information so quickly, and they can be sloppy and make mistakes.

In an exclusive article, published today by GigaOm, Derrick Harris explains how CrowdControl works to analyse the 'sentiment' behind crowd-sourced information by combining human and machine. (Still sounds scary? Keep going)

The program is based on Amazon's Mechanical Turk - an existing platform that effectively crow-sources crowd-source analysis, using over 500,000 workers in 190 countries to analyse social media.

CrowdControl is a program built on top of Mechanical Turk, which, according to Harris, uses "15,000 rules to determine how accurate workers are in completing their tasks". This allows it to manage the crowd and "identify best workers". Exactly how is does this is kept secret, but the process involves comparing the answers given by different people, asking workers questions that the program already knows the answers to and comparing the amount of time workers take to do their jobs.

In an explanatory video on the company website, CEO and founder of CrowdControl Max Yankelevick says the program allows for a "much higher data quality output and at much lower cost". It also saves time: "instead of taking weeks to analyse huge amounts of social media we can do it in minutes or hours".

It still sounds a bit dystopian. CrowdControl helps companies in: "deciding which workers are lying, which workers are telling the truth, which ones are good workers". As part of the system, accurate workers are rewarded, and inaccurate ones are penalized.

Harris writes that social media "sentiment analysis" is already becoming "big business" for firms like IBM and SAS. In this context - allowing marketers to see what people are saying on social media about their brands, for example - it has an obvious application.

However, when it comes to other uses of crowd-sourced material, especially journalism, man is always going to win over machine. In an article in Salon yesterday, Bonnie Stewart wrote that we should be "wary" of using social media as "an actually measure of value and influence". Writing about the influence-gaging program Klout, Stewart notes that numbers don't necessarily reflect the value of what people are saying online. Analytics make social media "all business", when in reality the value of social media originates in the fact that it allows normal people to build relationships. In the end, we shouldn't let "a metric to do a human's job".

Or, as Stewart puts it, not "unless, of course, you believe in a world where Justin Bieber is actually the most influential human alive."

Sources: CrowdControl, GigaOm, Salon