Statement from SODU2022

The third annual Scottish Open Data Unconference (SODU2022) took place in Aberdeen on 5–6 November 2022 in Aberdeen and online. Over the two days, attendees primarily from civic society, participated in 31 themed discussions. These covered diverse topics such as Open Data’s contribution to the economy; Policy, Strategy and Legislation of Open Data; The Technology behind Open Data Scotland; and The Ownership of Properties in Aberdeen City Centre. The organisers and participants wish to thank The Data Lab for their generosity in sponsoring the event, and ONE Tech Hub for providing the venue for the unconference.

The following statement was developed from the final session of the weekend. It has been collated and published on behalf of those at the unconference and in wider civic society by the organisers of the event, Code the City. 

Why open data matters

Making data (primarily government data) available under an open licence has a number of identified benefits. These may be categorised as humanitarian, social, economic, performance and environmental [1][2]. 

The Current State of Open Data In Scotland

While the Scottish Government publicly accepts the value of open data, the reality is that delivery on strategy and policy lags far behind what should be delivered. 

The introduction to the 2015 Open Data Strategy for Scotland [3] identifies many of the benefits and reasons for making data open. The Scottish Government’s participation in the international Open Government Partnership [4] is based on the social benefits of improving transparency and accountability, and enhancing citizen participation. The Scottish Government’s Digital Strategy 2021 [5] identifies the potential economic benefits of open data through innovation and entrepreneurship. In this context it is estimated that the value to Scotland’s economy of this, if done well, calculated as a percentage of GDP would be of the order of £2.23bn per annum. 

That said, apart from some small and isolated instances of very good open data publishing in Scotland the majority of public bodies (thought to number 179 but ironically there is no central set of open data to corroborate that) including health boards, local councils, universities, government departments etc publish no, or very little open data. 

It is not clear who is the lead for open data for the public sector in Scotland. Where does responsibility sit for implementation of the various strategies and plans? Whose role is it to monitor performance, report on that, or measure impact. And in each of the 179 or thereabouts public bodies, who has the responsibility to ensure that their organisation is implementing open data to deliver the multiple benefits noted above?

When open data is produced it often suffers from many issues:

  • Sporadic publishing
  • Lack of consistency or standards – for example across all 32 local authorities
  • Lack of current and up-to-date datasets  
  • Gaps in categories of published data by organisations  
  • Short-termism which sees projects commit £10,000s of funding then shutdown publishing platforms and delete the data

However, in civic society there is a small but thriving community for open data. They have, in addition to organising and attending three annual unconferences for Open Data, built, maintained and enhanced the Open Data Scotland portal [6].This project has provided a range of opportunities beyond making what open data exists in Scotland findable for the first time. It has also shared knowledge and expertise, and encourages community participation in the development of the site.

Threats or opportunities

Working with open data increases the breadth of experiences of students developers, analysts, businesses, and others, and increases opportunities. Participation in the development of projects such as Open Data Scotland continues to provide educational opportunities as well as societal benefits.

The lack of open data, provided to high standards, universally, and in a sustained manner is a barrier to innovation and a threat to transparency and accountability. 

Short term data publishing projects with no longevity are a risk to business continuity for those who are dependent on it.   

There is a broken feedback loop to publishers. Individuals and organisations, especially businesses, which are using open data need to be vocal about it. Otherwise, it is only natural for publishers to assume that their data is not being used, deem it to be without value, which may lead to termination of publication.

The government has created strategies and plans, but without mandate to publish for every public body. This has been shown over seven years not to work. [7]

There is a double threat to the provision of open data in Scotland. While the Scottish Government is refreshing its 2015 Open Data Strategy, the community fear that the new version will not be fully consulted on and that it will seek to water down the commitment which the Scottish Government made to data being made ‘open by default’. This would be a retrograde step and act against the intentions of the 2021 Digital Strategy and participation in the Open Government Partnership. 

In addition the Retained EU Law (Revocation and Reform) Bill [8] being put before the UK parliament would sweep away the UK’s Re-use of PSI Regulations 2015 on which basis data which has not been explicitly published as open data can be sought and used. As Jeni Tennison of ODI noted “[without these regulations] and with unprecedented pressures on the public purse, public bodies will start charging for data. Services using it will become more expensive. Some will close down. Others just won’t be built.” [9] This would kill open data in Scotland and stifle delivery of the Digital Strategy 2021

What Scotland Needs

We have identified that for Open Data to flourish in Scotland we need better commitment to, and engagement from, all organisations, bodies, or people with an interest in open data and the benefits it can deliver. We also have the following specific requests. 

From Government

We need oversight of delivery of open data in the Scottish Government to be a specific role – not be spread over multiple roles and structures. 

We need the refresh of the Open Data Strategy to be fully consulted on. It needs to strengthen, not weaken, the Scottish Government’s commitment to open data publishing. It needs to introduce regulations obliging each publicly-funded body in Scotland to deliver open data. It needs to identify  a specific role in each public sector organisation which will have a responsibility for implementing the strategy. 

It needs to provide a monitoring framework for ministers on progress in implementing the strategies which involve open data. This needs to be reported on publicly.

The Scottish Government needs to join with civic society in lobbying the UK government to retain R-PSI regulations and be prepared to introduce Scottish legislation to replace it if this fails. 

For businesses to use open data to deliver new products and services as outlined in the Digital Strategy, businesses need reliable, regular publishing without changes. Charging for data is a barrier to innovation and needs to cease. 

From Business

Anyone, including businesses, who are using open data need to be open about it. They should avoid agreeing contracts which oblige them to be silent about open data use to provide services which are only deliverable using open data. If Government is to be persuaded to make data available and keep making it available the use needs to be seen. We need businesses to join with civic society in lobbying for more, better open data.

From Education sector

We need Scotland’s schools, colleges and universities to use open data in their curriculum. We need teaching staff to better understand what open data is, how it can be used and to use it in course work. We need colleges and universities to publish open data which can be re-used too. 

From The Third Sector

The third sector needs to engage with civic society in promoting the use and benefits of open data. They need to demand that data is published openly which they can use. They need to educate their trustees and staff about the benefits of open data – such as the use of standards and platforms including 360 Giving.[10]

From The Press

We need those in the press to participate in the conversations about open data, to get involved with civil society groups, to challenge government and be involved with the broader community.  We need a press which understands the potential and power of open data and to help share the case for it and stories about open data. 

From Civic Society

We need the public to understand what open data is and what it can potentially deliver. We can help by providing access to education, resources such as the Open Data Scotland portal, and success stories from using open data. Government and the education sector have a responsibility here too. We need the public to tell us about challenges and success stories. 

[1]  https://www.europeandataportal.eu/en/training/what-open-data

[2] https://opendatahandbook.org/guide/en/why-open-data/ 

[3] https://www.gov.scot/publications/open-data-strategy/ 

[4] https://www.gov.scot/policies/improving-public-services/open-government-partnership/ 

[5] https://www.gov.scot/publications/a-changing-nation-how-scotland-will-thrive-in-a-digital-world/  

[6] https://opendata.scot 

[7] https://codethecity.org/2019/11/15/scotlands-open-data-february-2019-an-update/ 

[8] https://bills.parliament.uk/bills/3340 

[9] https://twitter.com/jenit/status/1576140360820330496?s=61&t=nI2hETRy_yo0zmhc13ycCg 

[10] https://www.threesixtygiving.org/ 

Header Photo by Hannah Busing on Unsplash

One week to SODU2022 – time to start thinking

Do you ever get the feeling that patterns are emerging, the stars aligning, things coming together, coincidences happening more frequently? Of course there is a lot of magical thinking behind those feelings – but the sessions I’ve attended, books I’ve read, speakers I’ve heard, conversations I’ve participated in this week all point to the importance and significance of SODU 2022 next weekend.

Mid-week I had a conversation with represenatatives from Scottsh Government about a forthcoming refresh of their 2015 Open Data Strategy. I was part of the group which drew up the original strategy. Between the drafting and the publication of final version, any sense that implementing the strategy was mandatory was lost. Some of us warned that this would lead to low adoption and therefore low impact in terms of data being released. We were, it is sad to say, proved correct. Of course the published strategy does say that Scotland’s data is to be open by default, but in practice it is anything but that.

In 2019 I estimated that over 95% of public sector data that should be open remained closed. And one look at the automatically-generated live table of local authority open data coverage shows what a long way there is to go!

While it is pleasing to hear that the Scottish Government is refreshing its open data strategy, the reason for that refresh is not, as far as I can tell, to make open data manadatory or to put measures in place to benchmark performance. As far as we can ascertain no-one monitors or has monitored the implementation of the 2015 strategy; even to the extent of counting as we did in 2019 the number of datasets released, itself a crude measure, And if I read the runes correctly, that commitment to data open by default might not make it in to the next iteration of the strategy which would be a major leap backwards. Civic society needs to make their feelings known about this in the strongest possible terms.

On Friday I gave a presentation to the University of Aberdeen’s Open Access Week’s Open Data event. I spoke about making open data useful, useable and used. The talk was one of half a dozen. I found the talks by each of the others speakers very good – but the one by Paola Masuzzo very impactful. The scenarios she spoke of in the struggle to get good government data resonated with the fight I and others put up to get Covid Data released in 2020. Paola is on the board of On Data and their website has a lot of great content. Unless you are fluent in Italian you might be better allowing Google to translate it if you open it in Chrome browser.

Another conversation that I have been having is with Jack and Karen, my colleagues in the Open Data Scotland project. This was born at Code The City hack weekends and previous SODU sessions. Should it remain as a CTC project? Should we spin it off as its own Community Interest Company? What is its role beyond being an open data portal? In my view we need to be more of a lobbying, pressure group for OD in Scotland. For example I believe that one week away from SODU2022 we have one person from the whole of the public sector in Scotland attending for only one day. There should at least be be someone there from every council health board, government agency etc. That would mean about 180 government attendees. But if their job doesn’t have open data at its core, and there is no obligation to publish, and no consequences to not doing so, why bother publishing? And why bother attending this weekend event?

It could be argued that since the Digital Strategy 2021 in Scotland does recognise the economic vlaue of open data then some of the core digital team with a responibility for implementing that should be there?

It was higlighted at SODU2021 and 2020, if I recall correctly, that there is a gap between government and civic society, and that the enormous goodwill and positivity of the civic group is a resource which government ignores and squanders. Chatting to someone else this week we puzzled to understand why the public sector is so afraid of engagement? Is there a sense that they know they are failing at OD and seek to avoid questioning? We know that several civil servants say they are committed to OD, but that there is no ministerial support in Scotland. Could that be it? We are a genuinely friendly community – and would welcome them with open arms.

This week I’ve been listening to the audio version of The Moves That Matter: A Grandmaster on the Game of Life by Jonathan Rowson, who I knew when he was a teenager in Aberdeen. At one point Jonathan said something about chess that I’ve long thought about applying to open data. He argues that chess “falls through the cultural cracks and lacks a stable cultural category that honours all of its elements; chess is more than a sport but is less than sport; chess is educational and educative but is not education; chess is full of artistic ideas but is not art; … is in many ways scientific but it not science“. He draws from this that without a cultural category that corresponds to a government department that would fund it, or a media section that would report it, it goes underfunded and unreported. The parallels with open data are deep. What IS open data? How do we get it funding and supported in `government? And recognition and attention?

Today was Wikidata‘s 10th birthday. I’m a big fan of Wikidata – and I train people to use it on behalf of Wikimedia UK. I also give guest lectures at RGU to Master students on the topic. We’ve used Wikidata in many projects at CTC hack events too. I’m convinced that Wikidata has a significant role to play as a place for publicly-curated linked open data – and filling in for missing ‘official’ government data,

Finally, I was contacted by someone who had attended the UK Open Data Camp that we hosted in 2018 in Aberdeen. He’s now living in Orkney and having heard about SODU2022 he’s making the journey down to attend, which is great to hear. We always look forward to catching up with the broader community.

Personally, I can’t wait to be at SODU2022. I hope that you do to and that we can have some meaningful discussions on the points above and much more!

If you haven’t booked yet you can still grab a ticket now.

[Published 29 Oct 2022. Edited 30 Oct to correct spelling errors and for readability.]

Header photo by israel palacio on Unsplash

SODU 2022

The Scottish Open Data Unconference (SODU) is back for a third year.

Since our inception in 2014 Code The City has consistently championed Open Data. Our trustees set up ODI Aberdeen, the only Scottish node of the Open Data Institute. As an organisation, and as individual trustees, we’ve worked to highlight the case to government for open data, to use open data, to make data open, and to educate others in the benefits and opportunities of open data. The highly-successful Open Data Scotland portal, built by a team of community volunteers to address a gap in Government provision, grew from conversations at previous SODU events and hack weekends which we hosted.

Why open data is important

Making data, particularly government data, open has very many benefits – from transparency to citizen empowerment; from providing fuel for innovation to improving government efficiency and much more. The Open Data Handbook is a great source of further background on this. In early 2020 the EU Data portal published a meta-study of the economic value of open data in early 2020. From this we extrapolated that the economic potential to Scotland, if OD publishing were done well, could be in excess of £2 billion per annum. Our trustee, Ian Watt published a research paper What is Open Data and Why Does it Matter? with the David Hume Institute which is available here under an CC-VY-SA 4 open licence.

A brief history of SODU

Back in 2020 we came up with the idea of a Scottish Open Data Unconference – an event which anyone with an interest in open data could attend (whether an activist or mildly curious; whether in Government, civic society, academia, or industry). The event was in part inspired by our hosting UK Open Data Camp in 2018 but our intention was that it would target our growing network. It wouldn’t just be about OD in Scotland but would look worldwide to help Scottish attendees develop their understanding of OD, create small networks to address challenges locally, help publishers connect with their data consumers and vice versa.

Then Covid got in the way – and our March 2020 event (for which we had 107 attendees signed up) was pushed back and run as an online event in September with just a third of that number. We repeated it online again in October 2021 with a similar number of attendees.

We’re now delighted that SODU 2022 is back as a physical event at the ONE Tech Hub, Aberdeen which is a great space for meeting and working together. We’ll also have high quality catering to look forward to! And we’re doing all we can to make it as safe as possible here.

What to expect

Based on previous attendance at SODU 2020 and 2021, and similar events we’ve run, we expect a broad mix of attendees. There will be a strong presence from civic society, with others from academia, the press and the IT / Data industry. W are also hopeful that we will see some attendees from government, given their policy / strategy / legislative obligation to publish data.

Each day will be different, depending on who is there and what they want to discuss (see format below). Some will attend to show their work or share research; others to run practical sessions. Some will offer informal educational slots; others will seek support for projects; and others still will seek or provide feedback on plans, or data provision.

The key to a successful event is the mix of attendees and people bringing a positive, open attitude, and a willingness to engage.

Format

This event will follow the open space format, where participants create the agenda each morning for the sessions that will happen that day. This is the time to pitch your questions to the many people who will be there, and to tell others about your project too.

Sessions in open space always work best when there is an interactive aspect so that the participants talk as much as the person chairing the session. This short video will tell you more about open space. The amazing thing is that it scales really well from 20 people to over 1000 people.

Booking


We’ve several types of tickets available.

  • Weekend-long ones for students at just £15,
  • General weekend tickets for just £25,
  • Daily tickets at £15 for either the Saturday or Sunday, and
  • Weekend online tickets for just £10.

And if the price is a barrier to your attendance just tell us and we’ll sort something out for you.

These events work best with a broad mix of attendees. No matter what your skillset, interest area or knowledge there is something for you – and something you can contribute. The write-ups of the 2020 and the 2021 events will remind you of how much can be achieved by a group of dedicated people over just a few hours!

Book here.

We hope that you can join us.

Ian, Karen, Andrew, Pauline, Bruce
Code The City
Charity SC047835

The Od-Bods project: update from CTC24

Why did we run this project? 

Theoretically, with the 2015 Scottish Government commitment to data being “open by default”, we should have universal publication of appropriate data as open data. In reality Scotland is very poorly served with Open Data. Few local authorities publish any, and those who do have little consistency. Beyond councils the picture is, if anything, even worse. Finding data is all but impossible. We set out to make data more findable, identify who is making data available and, perhaps as importantly, those who are not. We began with local government.

This work is a starting point, not an end point. 

Work done before CTC24

  • We wrote this blog post about this project to accompany the work done at CTC23, the forerunner to this event. 

What we achieved at CTC24, what impact we hope it will have

What challenges we have faced/are currently facing?

  • As always, lack of (good) engagement  with public sector.
  • There is no standardisation in how and where local government published its data.
  • Gathering data and cleaning it to output in a presentable format is currently a manual and laborious process.

What next – how can people get involved?

  • We have a page explaining the project, what our objectives are and what the plan is: https://opendata.scot/about/
  • There’s a big list of GitHub issues to be worked on here: https://github.com/OpenDataScotland/the_od_bods/issues 
  • The current milestone for Q1 2022 is to improve our data that we have gathered so far:
    • Fix known bugs with API calls
    • Tidy up inconsistent dataset tags
    • Identify and locate any missing data from the 32 local authorities which we haven’t found yet
    • Add more data features/metadata if possible

Join us at CTC25 to work on the project issues and work toward our next milestone! 

Modelling Government in Scotland on Wikidata

This was one of three projects which were worked on during CTC24 – Open In Practice. We asked Jan Ainali, who led the project, to explain it for those who were not present at the weekend event.

Why did we run this project? 

On Wikidata, there is a WikiProject for getting all the government agencies of all levels and the whole world properly modeled. Since it is a huge project, every way to try to break it down to bite size pieces are necessary. Work on the UK was already started, so it made a lot of sense trying to complete Scotland.

Work done before CTC24

In October during Scottish Open Data Unconference 2021 we got started on this task. We found some good sources and made fine progress, completing several categories of agencies. By completing here, I mean that we made sure there were items in Wikidata representing the agencies and that they were well enough modeled so that we could query for them. But we weren’t done, and some of the trickiest parts remained. 

What we achieved at CTC24, what impact we hope it will have

With a joint effort, we managed to sort out how the judicial system was organized, which was something that remained unclear since the last session. Most time was spent in researching to understand it, and when that was done, it was fairly straightforward to create items for the courts that were missing and to model the others in a way that made it possible to query for them. We also managed to sort out some other small tasks from the last event, and finally we could produce one huge query to get all Scottish agencies at once:  https://w.wiki/4TpN

Now we think this is the most complete and up-to-date list of Scottish agencies. If we are wrong about that, we would love feedback so that we can improve it.

What next – how can people get involved?

A few things are happening right now. First, we are importing agencies into the Govdirectory platform that is a more user-friendly view of Wikidata. We have already imported the local authorities, NHS boards and Health and Social Care Partnerships. You can find that data here: https://www.govdirectory.org/united-kingdom/

Since the large query was a bit messy, we will also try to improve the modeling in Wikidata. You are more than welcome to help with this. This will make queries for everybody simpler, and we will continuously be importing more agencies to the platform as we get done.

You can also help by adding contact points like email, official websites, social media accounts to the Wikidata items. You can either use the Wikidata button on the Govdirectory website, or you can go to the WikiProject page on Wikidata and run some queries there to find items to improve. https://www.wikidata.org/wiki/Wikidata:WikiProject_Govdirectory/United_Kingdom

Header Image by Jan Ainali, CC0, via Wikimedia Commons