Billy Tompson: Datablog: The government's new transparency intitiatives - what data will they release and how big a deal is it?

David Cameron has announced a raft of new data releases today. See what they are - and get the Guardian's expert assessment

David Cameron has written to each member of the Cabinet today pledging a whole raft of new data releases - announced on the new-look Downing Street website. (You can read the full text of his letter here).

These are just the latest in a series of government data initiatives, which have included releasing the Treasury's mighty Coins database, the senior civil service, spending over �25,000 and full details of ministerial hospitality.

They've been of varying success - Coins in particular was a textbook example of how not to do it. But today's announcements combine to arguably make the British government the biggest publisher of open data in the world - certainly data.gov.uk is now several times bigger than the US' data.gov.

So, what will be released? There are 20-odd commitments here, so we asked our specialist reporters what they think.

NHS

? Data on comparative clinical outcomes of GP practices in England to be published by December 2011, following the lead of the NHS in London which has agreed a set of 22 indicators with local GPs.
? Prescribing data by GP practice to be published by December 2011, as per the Growth Review.
? Complaints data by NHS hospital so that patients can see what issues have affected others and take better decisions about which hospital suits them. This commitment will be met by October 2011.
? Clinical audit data, detailing the performance of publicly funded clinical teams in treating key healthcare conditions, will be published from April 2012. This service will be piloted in December 2011 using data from the latest National Lung Cancer Audit, commissioned by the Healthcare Quality Improvement Partnership (HQIP) as part of the National Clinical Audit and Patient Outcomes Programme (NCAPOP).
? Data on staff satisfaction and engagement by NHS provider (for example by hospital and mental health trust) will be published from December 2011.
? Data on the quality of post-graduate medical education by provider from April 2012.

Guardian health editor Sarah Boseley says this is significant.

From the health perspective, this is going to be a really interesting exercise. The coalition has promised an openness and transparency about results that we have never experienced before, but there is no doubt it will be very hard to achieve anything meaningful that is of use to patients. The man pioneering this is Bruce Keogh, NHS medical director and former president of the Society of Cardiothoracic Surgeons - the medical speciality that tried harder than any others (most others have not tried) to compile outcomes data (in their case based on patients' death rates) and did an excellent job in the end - although the data is not easy for patients to find even now. He has been committed to encouraging, persuading and pushing other specialities to follow suit, but as a Guardian investigation into vascular surgery mortality rates showed last year, most are way behind.
Probably the most interesting data, though, will be that on GP practices. It has been known for many years that some are much better than others, but there have not been nationwide comparisons and certainly there has been little a patient could look at to make a choice of where to go. Keogh hopes to change all that. NHS London has attempted it, which gives the department of health something to work on, but the 22 indicators it will use will not be the same - discussions are going on with the Royal College of GPs to establish the most meaningful set.
All of these six initiatives are pilot schemes. We have a long way to go before there is a comprehensive set of data in the public domain that we can all scrutinise before we go to see the doctor and that will allow us to base our choice of hospital on something more than the distance from home and the cost of car parking - NHS Choices tries to help, but offers little more than reassurance that your local hospital is probably within acceptable boundaries on safety, cleanliness and efficiency. Many of us think we need much more detail than that.

Education & Skills

? Data enabling parents to see how effective their school is at teaching high, average and low attaining pupils across a range of subjects, from January 2012.
? Opening up access to anonymised data from the National Pupil Database to help parents and pupils to monitor the performance of their schools in depth, from June 2012. This will enable better comparisons of school performance and we will look to strengthen datasets in due course.
? Bringing together for the first time school spending data, school performance data, pupil cohort data and Ofsted judgements, from January 2012, in a parent-friendly portal, searchable by postcode.
? Data on attainment of students eligible for pupil premium to be published from January 2012.
? Data on apprenticeships paid for by HM Government, by organisation and by success rate to be published from July 2011.

Guardian education editor Jeevan Vasagar says

The government's commitment to offer parents a new single portal to compare schools offers a powerful new tool to guide families making choices about education. The site will bring together Ofsted judgements, data on school performance and spending, and information about the pupil body. This offers a more rounded portrait of a school's performance than its position in league tables. Depending on the level of data provided, parents will be able to judge how good the teaching is - both from the Ofsted report and data on how much progress children are making. Providing families with more easily accessible information is crucial to the goverment's ambition to drive up standards by increasing school choice. The idea of creating a "one stop shop" website of this kind was advocated in a recent report by the thinktank CentreForum. The thinktank recommended that it should include private schools too, to exert upward pressure on the quality of all schools and downward pressure on private school fees.

Criminal justice

? Sentencing data by court will be published by November 2011, enabling the public to see exactly what sentences are being handed down in their local courts, and compare different courts on a wide range of measures. The data, anonymised, will include the age, gender and ethnicity of those sentenced, the sentence given, and the time taken at each stage from offence to completion of the case in court.
? Data on performance of probation services and prisons including re-offending rates by offender and institution. To be published from October 2011.
? From May 2012, the national crime mapping website, Police.uk, will provide the public with information on what happens next for crime occurring on their streets, ie police action and justice outcomes.

Guardian home affairs editor Alan Travis says:

The decision to provide details on local crime maps of what action the police have taken and the sentences passed on the offenders responsible could finally fill a criminal justice information vacuum that has been created by the decline of local newspapers. But current plans seem to stop short of naming most offenders and only publicising their sentences for a maximum of 30 days. Ministerial hopes of putting photographs of local criminals on neighbourhood crime maps seem as far away as ever. The local court sentencing data could also help the public understand what is happening to crime. But the fact it will be anonymised means that most people's experience will remain reading media accounts of horrific crimes with little knowledge of what happened next.
Publishing reoffending rates for every prison and probation services will complement existing performance tables.

Transport

? In addition to opening up data owned by DfT and its arms length bodies, we are committed to working with the transport industry and data users to make public transport data open and freely available for re-use. Over the next year we will deliver:
? Data on current and future roadworks on the Strategic Road Network to be published from October 2011, and subject to consultation to extend this during 2012 to Local Authority Streetworks Registers maintained under statute.
?�All remaining Government-owned free datsets from Transport Direct, including cycle route data and the national car park database to be made available for free re-use from October 2011.
?�Real time data on the Strategic Road Network including incidents, speeds and congestion to be published from December 2011.
?�Office of Rail Regulator to increase the amount of data published relating to service performance and complaints by May 2012.
Rail timetable information to be published weekly by National Rail from December 2011.

Guardian industrial editor Dan Milmo says

Rail data will be consumed vigorously by a travelling public, and passenger watchdog, that believes commuters are not getting value for from money from ever-escalating fares that will continue to rise above inflation until 2015 at least

Government financial information

We are working with the purchase and payment card providers to provide a consistent method of reporting government procurement card spend data for transactions above �500 in value, so this is available for publication on departmental websites, from end September 2011.

Guardian Whitehall correspondent Polly Curtis says

Government procurement cards are held by 140,000 Whitehall officials who ran up a �25m bill last year. Bills released to The Telegraph under the Freedom of Informatin Act included apparently profligate spending such as a �370,000 bill for restaurants and take-aways, �3m on foreign travel and �117,000 for leisure activities including theatre tickets, golfing trips and football matches. The new publication, from September, of those bills will only be for items above �500 meaning they are unlikely to include some of the more controversial items of restaurant and hotel bills or gifts and flowers for staff.

Text of David Cameron's letter to the Cabinet

As you know, transparency is at the heart of our agenda for Government. We recognise that transparency and open data can be a powerful tool to help reform public services, foster innovation and empower citizens. We also understand that transparency can be a significant driver of economic activity, with open data increasingly enabling the creation of valuable new services and applications.
In May 2010, I set out the Government's specific commitments on transparency in a letter to Cabinet colleagues. Over the past 12 months, we have successfully fulfilled these commitments, and demonstrated global leadership in government transparency and open data.
I am writing to you today to celebrate our achievements over that period and set out new commitments for the next 12 months. These commitments represent the most ambitious open data agenda of any government in the world, and demonstrate our determination to make the public sector more transparent and accountable.
These achievements would not have been possible without the efforts of our advisory groups: the Public Sector Transparency Board and the Local Public Data Panel. The members of these groups have continuously encouraged us to be more radical, more ambitious and more committed to this work. I thank each member for their support.
If our transparency focus over the past 12 months has been to open up core central government data in areas such as spending, our priority over the next year will be to release new data on the performance of public services. This revolution in government transparency will make it easier than ever before for the public to make informed choices between providers and hold government to account for the performance of key public services.