Open Data Scotland – a nudge from OD Camp?

Note: This blog post was originally published in November 2018 at CodeTheCity.co.uk and was archived here with redirects from the original URL. 

Over the first weekend of November 2018, just over 100 people congregated in Aberdeen to attend the UK Open Data Camp. We’d pushed hard to bring it to Scotland, and specifically Aberdeen, for the first time. The event, the sixth of its type, which follows an unconference model where the attendees set the agenda, has previously taken place in England, Wales and Northern Ireland.

I’m not going to go through what we did over the weekend, you can find plenty of that here and here. There are links to all 44 sessions which took places on this Google doc, and many of those have collaborative notes taken during the sessions.

Instead this is a reflective piece, seeking to understand what OD Camp can show us about the state of Open Data in Scotland and beyond.

Who was there?

Of the 100+ attendees, including camp-makers, we estimate that about 40 were from the public sector. Getting exact numbers is hard – people register in their own name, with their own email addresses, but we think that is a good guess.

While this sounds good, during the pitching session on the first day Rory Gianni asked a question: “Hands up who is here from the Scottish public sector?” Two people’s hands went up out of 100+. Each were from local authorities, Aberdeen and Perth city councils, and a third (also from Aberdeen) joined later on Saturday.

This is really concerning and shows the gulf between what Scotland could, or rather must, be doing and what is actually happening.

The Scottish Public Sector

It is estimated that the Scottish Civil Service encompasses 16,000+ officers. It encompasses 33 directorates,  nine executive agencies  and around 90 Non-departmental public bodies (NDPBs) plus other odds and ends such as the Crown Office and Procurator Fiscal Service.

Then we have 14 health boards, 32 local authorities, 32 Joint Health and Social Care Partnerships and so on.

All of these should be producing open data.

Reality

Sadly, we are very far from that. Few are of any scale or quality. I’ve written about this extensively in the past including in this blog post and its successor post.

So, if we use attendance by the Scottish public sector, at a free-to-attend event which was arranged for them on their very doorstep, as a barometer of commitment to open data, it is clear that something is rotten in the state of Denmark Scotland.

Three weeks on

Since the event, I’ve reached out to the Scottish Government through two channels. I contacted the Roger Halliday, the Chief Statistician, the senior civil servant with a responsibility for Open data, and responded to a Twitter contact from Kate Forbes, the minister for Public Finance and Digital Economy.

I then had an hour-long conversation with Roger and two of his colleagues. This was a very positive discussion. I took away that there is a genuine commitment to doing things better, underpinned by a realism about capacity and capability to widely deliver publication and engagement with the wider OD community. I have agreed to be part of a round table meeting on OD to be held in the new year – and have expressed a commitment to assist in any way needed to improve things.

Meanwhile

Ironically, in the midst of this three week period, the Scottish Government published its Open Government action plan. This emerged on 14th November and is open for feedback until 27th November. So, if you are quick, you can respond to that – and I encourage you to do so. While this certainly seeks to move things in the right direction in terms of openness and transparency, it is extremely light on open data and committed actions to address some of the issues which I have raised.

My next blog post will be a copy of the feedback which I provide, and on which I am currently working.

And finally

When I started drafting this post I was in a very negative frame of mind as regards the Scottish Open Data scene – and particularly in terms of the public sector. In the intervening period, I  launched the Scottish Open Data Action group on Twitter. The thinking  behind this was to get together a group of activists to swell the public voice beyond mine and that of ODI Aberdeen.

Given the way things are moving on with the Scottish Government and the positive engagement that has begun, the group, which is in its infancy, may not be needed as a vocal pressure group. Instead we could be a supportive external panel who provide expertise and encouragement as needed. Who knows – let’s see!

It is easier to recycle a fridge than reuse Scottish public sector website content and data!

During the course of  Code The City 17: Make Aberdeen Better this weekend we made a startling discovery. It is easier to recycle your old fridge-freezer than to get data and content for re-use from Scottish public sector websites. As a consequence, innovating new solutions to common problems and helping make things easier for citizens is made immeasurably more difficult.  

One of the event’s challenges posed was “How do we easily help citizens to find where to recycle item ‘x’ in the most convenient fashion. That was quickly broadened out to ‘dispose of an item” since not everything can be recycled – some might be better reused, and others treated as waste, if it can’t be reused or recycled. With limited kerbside collections, getting rid of domestic items mainly involves taking them somewhere – but where?

With climate change, and the environment on most people’s minds at the moment, and legislative and financial pressures on local authorities to put less to landfill, surely it is in everyone’s interest to make it work as well as it can.

To test how to help people to help themselves by giving advice and guidance, we came up with a list of 12 items to test this on – including a fridge, a phone charger, a glass bottle, and tetra pack carton. On the face of it this should be simple, and probably has been solved already.

The Github Repo

All of Code The City hack weekend projects are based on open data and open source code. We use Github to share that code – and any other digital artefacts created as part of the project. All of this one’s outputs can be found (and shared openly) here.

Initial research

That was where we started: looking to see if the problem has already been solved.  There is no point in reinventing the wheel.

We looked for two things – apps for mobile phones, and websites with appropriate guidance.

Aberdeen specific information?

Since we were at an event in Aberdeen we first looked at Aberdeen City Council’s website. What could we find out there?

Not much as it turned out – and certainly not anything useful in an easy-to-use fashion. On the front page there was an icon and group of suggested services for Bins and recycling; none of which were what we were looking for.

ACC Bins and recycling
ACC Bins and recycling

Typing recycling into the search box (and note we didn’t at this stage know if our hypothetical item could be recycled) returned the first 15 of 33 results.  As shown below.

Search results for recycling
Search results for recycling

The results were a strangely unordered list – neither sorted alphabetically nor by obvious themes. So relevant items could be on page 3 of the results. Who wants to read policies if they are trying to dispose of a sofa? Why are two of (we later discovered) five recycling centres shown but three others not? Why would I as a citizen want to find out about trade waste when I just want to get rid of a dodgy phone charger?

Why is there a link to all recycling points (smaller facilities in supermarket carparks or such like, with limited acceptance of items), but apparently not to all centres which cover much more items? Actually there is a link ‘Find Your Nearest Recycling Centre’ (but not your nearest recycling point which are much more numerous). This takes you a map and tabular list of centres and what they accept. And it is easy to miss the search box between the two. No such facility exists for the recycling points.

Open Data?

Perhaps there is open data on the ACC Data portal that we could re-purpose – allowing us to build our own solution? Sadly not – the portal has had the same five data sets for almost two years, and every one of those has a broken link to the WMSes.

If we were in Dundee we could download and use freely their recycling centre data. But not in Aberdeen.

Dundeee recycling Open Data
Dundeee recycling Open Data

Apps to the rescue?

There are some apps and services that do most of what we are trying to do. For example iRecycle – Iphone and Android is a nice app for Android and iOS that would work were it not for US locations only.

We couldn’t find something for Scotland that worked as an App.

Other sources of information?

Since we drew a blank as far as both Aberdeen City Council and any useable apps, we widened our search.

Recycle For Scotland

The website Recycle For Scotland (RFS) is, on the face of it a useful means to identify what to do with a piece of domestic waste. Oddly, there appears not to be any link to it that we could find from any of the ACC recycling pages.

BUT …… it doesn’t work as well as it could and the content, and data behind it have no clear licence to permit reuse.

The Issues with RFS

Searching the site, or navigating by the menus, for Electrical Items results in a page that is headed “This content was archived on 13th August 2018” – hardly inspiring confidence. No alternative page appears to exist and this page is the one turned up in navigation on the site.

Recycle For Scotland Archived content
Recycle For Scotland Archived content

Searching for what to do with batteries in Aberdeen results in a list of shops at least one of which closed down about 18 months ago. Entering a search means entering your location manually – every time you search! This quickly becomes wearing.

While the air of neglect is strong, the site is at least useful compared to the ACC website. But it doesn’t do what we want. Perhaps we could re-use some of the content? No – there is no clear licence regarding reuse of the website’s content.

The site appears to be a rebadged version of Recycle Now, built for Zero Waste Scotland (ZWS). According to ZWS’s Terms and Conditions on their own site, and deeply ironically, you can’t (re)use any materials from that site.

Zero Waste Scotland - zero re-use
Zero Waste Scotland – zero re-use

ZWS are publicly funded by the Scottish Government and the European Regional Development Fund – all public money.

Scottish Government Fund ZWS
Scottish Government Fund ZWS

Public funding should equal open licences

We argue that any website operated by a government agency, or department, or NDPB, should automatically be licensed under the Open Government Licence (OGL). And any data behind that site should be licensed as Open Data.

The Scottish Government’s own website is fully licenced under OGL.

Changing the licensing of Recycle For Scotland website, making its code open source, and making its data open would have many benefits.

  • its functionality could be improved on by anyone
  • the data could be repurposed in new applications
  • errors could be corrected by a larger group than a single company maintaining it.

Where did this leave us?

Having failed to identify an app that worked for Scotland, nor interactive guidance on the ACC website, we tried the patchy and, on the face of it, unreliable RFS site. We’d turned to the data and whether we could construct something useable from open data and repurposed, fixed, content over the weekend – this is a hack event after all.

But in this we were defeated – data is wrapped up in web pages: formatted for human readability, not reuse in new apps.

Websites which were set up to encourage re-use and recycling ironically prohibit that as far as their content and data is concerned, and deliberately stifle innovation.

Public funding from the City Council, the Scottish Government and the European Regional Development Fund is used to fund sites which you have paid but elements of which you cannot reuse yourself.

Finally

At a time of climate crisis, which the Scottish Government has announced is a priority action, it can’t be right that not only is it difficult to find ways to divert domestic items from landfill,  but also that these Government-funded websites have deliberate measures in place to stop us innovating in order to make access to reuse and recycle easier!

Hopefully politicians, ministers and councillors will read this (please draw it to their attention) and wake up to the fact that Scotland deserves, and needs, better than this.

Only by having an Open Data by default policy for the whole of the Scottish Public Sector, and an open government licence on all websites can we fix these problems through innovation.

After all if the non-functioning Northern Ireland Assembly can come up with an open data strategy that commits the region to open data by default, why on earth can’t Scotland?

See below:

“Northern Ireland public sector data is open by default. Open by default is the first guiding principle that will facilitate and accelerate Open Data publication.”

NI Open Data principles
NI Open Data principles

[Edit – Added 12-Nov-2019]

Postscript

If you are interested to read more about the poor state of Scottish Open Data you might be interested in this post I wrote in February 2019 which also contains links to other posts on the subject:

Scotland’s Open Data, February 2019. An Update.

Sadly, not much has changed in the intervening nine months.

[/Edit]