More than half the countries with freedom of information laws in place do not follow them, the Associated Press has found in an 11-month investigation into citizens' rights to know what their governments are doing.
Having effectively used FOI requests in investigative stories in the US, especially at the state and local government level, the AP decided that such requests were a tool which could be better taken advantage of at an international level, said John Daniszewski, vice president and senior managing editor for international news.
In part as an attempt to react to the 10th anniversary of the 2001 terrorist attacks in a constructive manner, the AP set about asking the European Union and 105 different governments around the world questions about how many arrests and convictions for terrorism there have been in the ten years since 9/11.
"I don't think many people even knew that 105 countries had these laws on the books," said Daniszewski, "and in some of the countries in which we used them they were virtually unused."
The study hopes to educate both AP journalists and other media organisations that these laws exist, and encourage people to "challenge governments to live up to the promise of these laws to provide open information to the public, in the public interest," he added.
Coordinated by AP lawyer and FOI specialist Karen Kaiser, more than 120 journalists worked on the study. It was a big logistical effort, Daniszewski said, and there was a considerable range of difficulty in filing requests.
Just 14 countries answered in full within their legal deadline, with newer democracies proving more responsive than some developed ones. "It was surprising how law-abiding and conclusive some under-developed countries were," Daniszewski said. "It was often the newer democracies who were more responsive than the big more established democracies," he added. Guatemala, Turkey, India, Mexico and Peru all replied swiftly and fully.
The AP highlighted Mexico's FOI law as a potential model: it allows anonymous requests via a website and all requests are acknowledged immediately, with full answers arriving in a month, published online, available to the public. The law, which took effect in 2003, has now seen 3,012 requests and 2,460 responses.
Thirty-eight further countries eventually answered most questions.
African countries had the worst response record - out of the ten countries which have passed right-to-know legislation, seven did not respond at all to requests, and only South Africa and Mozambique provided any useful information. The law in Uganda, which was passed 2005, seems particularly ineffective, with fees of about a week's wages required to even place a request and one journalist struggling for several years to obtain information, the AP explained.
The US - which was the third country to introduce FOI laws, in 1966 - proved less than helpful at first, directing the journalists to websites and providing only heavily redacted documents. Eventually the information was provided, but the AP specified in its story that it is still waiting on a 10-year-old request to the US State Department for information about a now-defunct Greek terror organisation.
Part of the problem is that the law was created at a time when all records were on paper. One of the advantages that newer democracies have is that their laws could immediately be made appropriate to the digital age.
Daniszewski said he didn't think that laws in many countries had been sufficiently digitally adapted, and that it would be much more efficient for citizens if they were offered the possibility to file a request via email or an online form.
Several countries have passed FOI laws effectively for financial gain, but the AP study found that those laws do not work as well as those adopted in response to public pressure. China implemented a system to allow citizens to make requests for public records in order to become a full member of the World Trade Organisation. Although the Chinese government has become somewhat more open, the AP reported that it never provided the information requested.
The AP also spoke with former UK prime minister Tony Blair, who lamented pushing through right to information laws in 2005. "You can't run government without being able to have confidential discussions with people on issues that are of profound importance," he told the AP in an interview. "What happens in the end is that you make politicians very nervous of actually debating things honestly, because they're worried about what's going to happen when there's a FOI request."
Can government secrecy be defended on these grounds? As far as journalists are concerned, no. As Daniszewski said, "more openness is always better. Governments of course will always have secrets related to security or to protect privacy, but I think the general rule is if they do things in the name of the public and tax payers then these people should have the right to know what their government is doing."
In the spirit of openness, the AP has made all the responses public, as well as the methodology of the questioning and a summary of countries' responses, using DocumentCloud. All of the AP's coverage can be found via its Facebook page and the organisation is inviting readers to suggest the sort of information they would like to get from a government agency anywhere in the world.
Will the study inspire more journalists around the world to question their governments using freedom of information laws? Advances in data journalism techniques mean that once a news organisation has obtained government data, it is ever easier to analyse. But without effective laws in place that governments actually respect, vital information will continue to be hidden.