While most FOI requests are relatively mundane, sometimes decisions on whether to make information public can have global ramifications that could cost lives

They are the photos the world will never see.

Hundreds of images of Iraqi, Afghani and other detainees in US military custody being abused in circumstances said to be worse than those depicted in the notorious Abu Ghraib photos that shocked the world.

During his election campaign, then Senator Barack Obama had pledged he would release the photos in their entirety.

Upon taking office and viewing the photos then listening to advice from government officials and military officers, Obama reversed his position and chose to keep the images secret.

For many of Obama’s idealistic supporters, the about face was the start of a series of disappointments in a man in whom they had placed so much trust. The message they took from the decision was that Obama was just another politician, that it was simply business as usual in Washington.

Professor Dan Metcalfe is one of the few people to have seen the pictures. And as director of the US Justice Department’s Office of Information and Privacy, he advised President Obama not to release them. There is an irony in this given Metcalfe has dedicated a long and illustrious career to advancing the cause of freedom of information. But it also neatly encapsulates the reality that not all information held by governments can, or even should, be accessible to all who want it.

The photographs in question were collated by US investigators in the wake of the Abu Ghraib scandal to try and find what other images might exist and possibly leak out causing the US further embarrassment and crucially, act as recruiting sergeant for various resistance organisations in Iraq and Afghanistan and even al-Qaeda itself.

Metcalfe explains it was exactly these fears that led him to recommend the photos should not be released.

“I happen to know more than a little bit about the photographs on which President Obama changed the position last May and that is because I consulted on the case for the Joint Chiefs of Staff back in 2005 and I am the one who, for better or for worse, came up with the legal position that was used in the courts and that was rejected by the 2nd Circuit Court of Appeals and the question was would the Government petition the Supreme Court and our new Solicitor General decided in April not to petition and that decision was reversed by President Obama, very, very, very unusual and it was reversed after he looked at the actual photographs.

“He reached the judgement, as did I, that the photographs were of such character that one could reasonably imagine that their disclosure would incite violence that would pose a very serious risk of death in the climate in western Asia. That’s a judgement that I reached and I had the advantage of having viewed those photographs and it was when President Obama viewed them in May that he did the remarkable thing of turning it around. The thing I must say is that the particular photographs in question are not like the nature of the Abu Ghraib photographs but the photographs, and the best way I can put this publicly because I am under a moral obligation flowing from my employment, if you were a young Muslim male and you were incensed to begin with by the US presence in your part of the world that if you saw these photographs and saw the US Government had taken them, you would be incensed many fold and be logically inclined towards violence.” But Obama’s decision not to release the images speaks to an inevitable truth about the operation of FOI: that nearly every opposition party promises greater transparency when campaigning for government and those self same parties tend to resile from their commitments once they do take office.

Metcalfe says: “As someone who has observed government since I was a teenager, I first started working for the justice department when I was 19, it is very tempting to promise more than you can deliver. There are two reasons for that.

One, you have a tremendous incentive to do so as a candidate and two, you don’t bear the burden of the decision making yet. You don’t know the extent to which what you are saying is unrealistic. You really don’t know how unrealistic it might be until you bear that responsibility. The second thing to say about that is that it is true in general to say of all sorts of policies and government issues but it is unquestionably true of the freedom of information area.

“As a government comes into power it is relatively easy to be in favour of openness for existing records because who created those records? It wasn’t you. Then time goes by and all of a sudden more and more, a greater percentage of those requests apply to records that you have created and the calculus shifts a little bit and you know what, I can tell you from the vantage point of many years of history that it doesn’t matter whether it is a seemingly conservative Republican administration or a seemingly liberal Democratic administration, it works just the same, it is human nature, the nature of the beast.” But Metcalfe, now Executive Director of the Collaboration on Government Secrecy at the Washington College of Law, says that despite this, FOI in the US has progressed steadily from its inception and that he expects that to continue under Obama.

He says: “In the big picture it has certainly progressed since the very beginning. The law was enacted in 1966 and took effect in 1967 but it didn’t really move forward with pace and vitality until seven years after that in 1974 [when] our then President Nixon committed a number of abuses that led to widespread government reforms including a major amendment to our Freedom of Information Act. We call those the 1974 amendments.

“That is when our FOI Act took off, so to speak. I saw the progression of it and I started working with the Act in 1976 as a young attorney and then I was appointed to be the founding director of the governmentwide policy office in the first year of the Reagan administration after litigating as a barrister for four years.

“There has been quite a variation from one political administration to another. Certainly there were the eight Reagan years and the elder Bush thereafter; those were pretty much of a piece and then a very different policy, far more disclosure oriented for the eight years of President Clinton. And then the pendulum proceeded to swing in the opposite direction under Junior Bush and if you take it to the current administration, we are moving back to where things were under President Clinton. Part of the problem is that for some people’s tastes we are moving too slowly back to where things were.”