Category: Policy

Note: This blog post originally appeared on codethecity.co.uk in November 2018 and has been archived here with a redirect from the original URL. 

The Scottish Government published its draft action plan on 14th November 2018. You can find it here. They are seeking feedback before the 27th November 2018.

Here is my feedback which I sent on 25th November.

Thank you for the chance to feed back on the drafts of the Scottish Open Government Action Plan and Commitments.

These documents are welcome and while they certainly set a path for moving Scotland further in the right direction in terms of openness and transparency, we should remember that those should not be our only aims. We need to ensure that we also address the need to use data and information to fuel innovation, and deliver societal and economic benefits for Scotland.

I have set out below my observations and suggestions in a number of areas which range from the general to the specific.

The public good

Data and information held by the Scottish Government and the public sector should be considered a Public Good. See https://www.nic.org.uk/wp-content/uploads/Data-for-the-Public-Good-NIC-Report.pdf and https://www.gov.uk/government/publications/data-for-the-public-good-government-response/government-response-to-data-for-the-public-good.

To deliver that public good requires freeing up information and data as a matter of course, rather than by exception.

There is one simple thing that could be done with immediate impact, and minimal effort, to free up large amounts of data and information for public re-use: adopt an Open Government Licence (OGL) for all published website information and data on the Scottish Government’s website(s), and other public sector sites, the only exception being where this cannot legally be done, as would be the case when personal data is involved.

The ICO’s own website (http://www.itspublicknowledge.info/home/TermsAndConditions.aspx) takes this approach: “Where the Commissioner is the copyright holder, information is available through the Open Government Licence. This means you have a worldwide, royalty-free, perpetual, non-exclusive licence to use the information, subject to important conditions set out in the licence.”

At present, websites operated by Scottish Government, local authorities, health boards etc.  all appear to have blanket copyright statements. I certainly could find no exception to that. With OGL-licensed content, where data is not yet available as Open Data (OD), a page published as HTML could be legitimately scraped and transformed to open data by third parties as the licence would permit that. Currently pages such as this list of planning applications, https://publicaccess.aberdeencity.gov.uk/online-applications/simpleSearchResults.do?action=firstPage contain valuable data but are caught by default, site-wide copyright statements.

Of course, in reality citizens, companies, universities and organisations do scrape website content, but it is done under the radar. This approach results in repeated scraping as the results are not published as open data, and there is consequently limited public benefit. Switching the licensing model to OGL by default, and copyright by exception,  would solve this and encourage both innovation and engagement: moving a supplier / consumer relationship to one where data and information are a shared public good.

The Scottish Government should mandate this approach not just for the whole of the public sector but also for companies performing contracts on behalf of Government, or who are in receipt of public funding or subsidy.

Targets for publishing

The Scottish Government’s own Open Data Strategy 2015 commits it to publishing data openly but despite my efforts and those of other contributors to it, the strategy mostly lacks hard targets, and sets overly-modest goals: “The ambitionis for all data by 2017 to be published in a format of 3* or above.”  One could ask if all of Scottish Government’s data wasactually published to 3* standard by the end of 2017. If not, how much? Who knows – is this even measured, reported on or published?

Therefore, any new action plan should have harder, more specific targets. It is arguable that the lack of these, and of a clear Open Data Policyfor Government, as I called for in 2015, allows overly-pressed civil servants to have much less focus on publishing open data than is needed, resulting in inadequate resources being applied to that. So, ideally this action plan should be underpinned by policy for the whole of the Scottish public sector to ensure that effort and resource can be targeted on publication.

To support this, the public benefits of open data publishing, both in social and economic terms, should be made clear to all data publishers.

Every FOI request should be assessed on receipt, identifying whether it is for data or whether data publishing would satisfy that and future similar requests. If so, the data set should be set for publication as OD with regular periodic updates.

Statutory obligations

I looked for, but could not see, in the action plan and other document, an acknowledgement  of the current statutory obligations on the Scottish Government in this area. Recognising, noting and commenting on these in the document would be a useful reminder of specific existing obligations but would also strengthen broader arguments for OD. The following list is not exhaustive.

There are obligations under the G8 Charter on Open Data https://www.gov.uk/government/publications/open-data-charter.

Further, there are existing clear obligations under The Re-use of Public Sector Information Regulations (2015) https://www.legislation.gov.uk/uksi/2015/1415/contents. There is a handy guide here:

http://www.nationalarchives.gov.uk/documents/information-management/psi–guidance-for-public-sector-bodies.pdf (see pages 22 onwards in particular).

Where specific legislation mandates open publication then this should be made clear, as is the case, for example, under The Public Services Reform (Scotland) (2010), if only to avoid this type of headline: https://www.heraldscotland.com/news/17238918.snp-ministers-missing-their-own-transparency-target/

Another example is the OECD’s “Compendium of good practices on the publication and reuse of open data for Anti-corruption across G20 countries: Towards data-driven public sector integrity and civic auditing”.

https://www.oecd.org/gov/digital-government/g20-oecd-compendium.pdf

Recommendations and best practice

There are many resources available online which demonstrate best practices which Scotland’s public sector should adopt in order to deliver the aims of the action plan. Again, these should be mandated for adoption in the action plan. Some examples follow.

Discoverability

A key part of publishing information and data openly is discoverability. To do this well means understanding and applying best practices. Having standard identifiers, descriptors, taxonomies etc. will aid discoverability.  So, all information and data publishing should use best practice, using the correct metadata and appropriate standards such as DCAT / DCAT-AP / DCAT.json.

There are some useful resources to assist in this such as

• https://guidance.data.gov.uk/dcat_fields.html
• https://digitalblog.ons.gov.uk/2017/01/13/describing-ons-datasets-with-standard-vocabularies/
• https://digitalblog.ons.gov.uk/2017/01/06/some-open-data-publishing-principles/and
• http://lga.learndata.info/#/id/587e3feeb61c46e176e7e429

The Scottish Government has an internal expert on this, who sits on the international standards board. It is imperative that his input is sought, and implemented rigorously, in terms of this application of standards.

Data as infrastructure

We should acknowledge the concept of data as infrastructure. See https://www.nic.org.uk/wp-content/uploads/Data-As-Infrastructure.pdf and https://theodi.org/topic/data-infrastructure/. Publishing to our best ability, based on standards and best practice will allow new products and services be developed for societal and economic benefit, and support innovation.

Reference Data

By using standard identifiers for things, such as UPRNs for properties, USRNs for roads and so on, data from multiple government sources can be aggregated about that object, and we can link items with certainty. If the identifiers are then made public, external data such as those from the private sector, can be amalgamated. There must be a concerted effort to make these identifiers public and re-usable. Instead of what appears to be a starting position of “we can’t do this because of x ” we must shift to “how can we do this and how can we sweep away barriers?” Where no identifiers exist for a specific domain, but it is identified that there would be benefit from having them, these should be created.

General approach to open data

Open Data is not a separate thing or process. The curation, management and publication of data is a continuum starting with the internal processes of the organisation. OD should be seen as the natural end point for all data where it is appropriate to publish openly. By adopting an open data by default approach, as outlined here  https://en.wikipedia.org/wiki/Open_by_default effort is expended on publishing, not on finding a reason or way to publish: data will be published as OD unless there are specific legal reasons why it can’t be. There are additional benefits to this, including improvements in data quality, de-duplication  and re-use of data internally by other departments or services.

Further, while the draft action plan focuses on statistical data, it needs to be recognised that while publishing statistical data openly, the scope needs to be so much wider: encompassing all branches of the Scottish Government, its directorates, its NDPBs, and other agencies. SG also needs to act as a leader to health boards, local authorities, and to joint health and social care partnerships, and work with others such as Scottish Cities alliance where work is ongoing.

We need to open up reference data, geographical boundaries, transactional data, financial data, in fact anything that need not be closed by default.

National portal

Scotland lacks a national open data portal. While this is not a necessity, in order to aid discovery, it would be an advantage, particularly when we have a growing number of existing places where data is being published across Scotland. Many other countries have national portals (https://www.opendatasoft.com/a-comprehensive-list-of-all-open-data-portals-around-the-world/ ) and some such as Austria have had a federation model of publishing at various levels of government in place for many years. If we get discoverability right, and tools such as Google’s data search engine (https://www.google.com/publicdata/directory) begin to mature, this may be less of an issue.

Geospatial commission

Both the recently-formed geospatial commission and the rapidly changing stance of Ordnance Survey is going to impact on what we can publish – with barriers being removed. This increased liberalism will mean that data which we could not publish 3 months ago will suddenly be publishable. Scottish Government need to be on top of that and acting on it to push out data as soon as it can. Beyond that, they should be routinely pushing OS on issues such as derived data to ensure that barriers to publishing are actively removed. Similarly, if reference data is opened up at a UK level, then the Scottish portion of that data needs to be highlighted by the Scottish Government.

Community Building

The action plan must include commitments to work with the Open Data community in Scotland. It is smaller than it should be since there has been relatively little data of value to work with up to now. Contrast with the position of Transport For London, one single organisation, whose open data as far back 2013 was reported to be responsible for 5,000 developer jobs and 500 apps. The Scots Govt needs to grow the OD community and develop it by being an active part of it; to actively seek input on what data sets would be most useful, to use the community as a sounding board; to gain the trust and support of the community by empowering them to be infomediaries who will build and develop products and services which enable citizens to use the data produced, and make sense of it.

Supporting education

Finally, the publication of open data needs to be seen as an educational resource too. Data should be available for use by schools, colleges and universities. Curricular development should encompass the use of open data. Outreach should work with teachers and lecturers so that children can understand their locality by using data pertinent to them. Honours-year and post-grad students in computing sciences should use open data in their projects. Innovation and entrepreneurship courses should encourage the use of public data. Journalism courses should teach data journalism, and so on.

Ian Watt

etc