Author: ianwatt

The Od-Bods project: update from CTC24

Why did we run this project? 

Theoretically, with the 2015 Scottish Government commitment to data being “open by default”, we should have universal publication of appropriate data as open data. In reality Scotland is very poorly served with Open Data. Few local authorities publish any, and those who do have little consistency. Beyond councils the picture is, if anything, even worse. Finding data is all but impossible. We set out to make data more findable, identify who is making data available and, perhaps as importantly, those who are not. We began with local government.

This work is a starting point, not an end point. 

Work done before CTC24

  • We wrote this blog post about this project to accompany the work done at CTC23, the forerunner to this event. 

What we achieved at CTC24, what impact we hope it will have

What challenges we have faced/are currently facing?

  • As always, lack of (good) engagement  with public sector.
  • There is no standardisation in how and where local government published its data.
  • Gathering data and cleaning it to output in a presentable format is currently a manual and laborious process.

What next – how can people get involved?

  • We have a page explaining the project, what our objectives are and what the plan is: https://opendata.scot/about/
  • There’s a big list of GitHub issues to be worked on here: https://github.com/OpenDataScotland/the_od_bods/issues 
  • The current milestone for Q1 2022 is to improve our data that we have gathered so far:
    • Fix known bugs with API calls
    • Tidy up inconsistent dataset tags
    • Identify and locate any missing data from the 32 local authorities which we haven’t found yet
    • Add more data features/metadata if possible

Join us at CTC25 to work on the project issues and work toward our next milestone! 

2021 – A Year in review

The continued running of hack weekends as online events due to the ongoing COVID-19 situation resulted in the widening of our attendee audience beyond Aberdeen. This saw attendance across the length and breadth of the UK and international attendees from the Netherlands, Sweden and Switzerland. These international attendees contributed to projects that had the potential to benefit the citizens of Aberdeen that our remit looks to support. This demonstrated that our message and vision is spreading beyond our traditional geographical base. However our attendance numbers for online-only events have dwindled compared to those held in physical spaces and we look forward to moving to hybrid events as soon as we can. 

We’ve supported the creation and initial deployment of two significant infrastructural projects: Open Data Scotland and Open Waste UK both of which will fill large gaps in society’s access to data about government.

We recently appointed two new trustees, Pauline Cairns and Karen Jewell who are already injecting additional enthusiasm and energy into our operations.

Events

CTC22: The Environment

Online – 20-21 Mar 2021

Event page

Key projects:

  • Bioregional Dashboards
  • Waste Wizards
  • Meet Your Next MSP

Notable Outcomes:

  • The website  https://openwastemap.uk was launched. This was underpinned by Open Streetmap data and 922 HWRCs which had been added to WIkidata. 
  • The Bioregion Dashboard team worked to make it easier for anyone to make a dashboard for their own area. 

CTC23: The Future of the City

12-13 Jun 2021

Event page

Key projects:

Notable Outcomes:

  •  The first steps taken to catalogue open data from local authorities in Scotland. Scripts written to gather data from council open data portals and GIS publishing platforms. Analysis of the status quo were performed. 
  • Prototyping of a sound map of Aberdeen.

CTC24: Open in Practice 

Online – 27/8 Nov 2021

Event page

Key projects:

Notable Outcomes:

Impact from pre-2021 Events

2021 saw the continuation of several projects that had been initiated at CTC events from previous years, a perfect demonstration of how CTC has a lasting positive impact.

Harbour Arrivals Transcriptions

During 2020 the Harbour Arrivals physical records for the period 1914 to 1920 were transcribed into a spreadsheet with a website built to make the data open for anyone to explore and use. In 2021 a data story was created focusing on the type of ships coming into the harbour from various ports, the imports to sustain the region, and the comments recorded by harbour staff.

SODU 2021

The second iteration the Scottish Open Data Unconference

2-3 Oct 2021

Event page

Key items:

  • While we had great engagement from civic society once again, there was poor showing from Government at all levels, Academia and Journalism.
  • Good news stories in Open Data
  • Attendees from overseas and across UK
  • Demonstration of Data Commons Scotland project

Notable Outcomes:

  •  A plan to take forward a number of existing open data projects including
    • Open Data Scotland
    • Wikidata representation of government in Scotland

Python Aberdeen

We held 11 Aberdeen Python User Group sessions over the last year with 10 speakers and a total of 240 attendees.

Non-event Based Achievements

Intern Project – Summer 2021

We took on our first intern this summer. Sara, a post-grad student from Edinburgh University worked with our trustee Ian, a group of volunteer transcribers, and Aberdeen City and Aberdeenshire Archives to open up data and images from the Aberdeen Register of Returned Convicts 1869 to 1939. This resulted in structured open data for 248 convicts being made available, as well as 59 images of them being licenced for reuse. You can read more on the project page.

Award nominations

For the second year running Ian received an honourable mention in the UK Wikimedian of the Year awards.

Code The City were delighted to be nominated in the Data category at the Open UK awards in November 2021. Unfortunately we were runners up to OKFN. 

Looking ahead

2022 will see a renewed strategy and business plan for CTC. We will return to Hybrid events as soon as we can safely do so. We will continue to support and build on projects such as Open Data Scotland. We will look at hosting one or more interns again this year. We will continue to seek to better commitment from government on the provision of open data – and get participation in events such as SODU.

Thank you

Thanks to all of our attendees, supporters, volunteers and interns who make our events so successful and impactful. We wish you a quiet and restful festive season and look forward to working with you again in the new year.

Ian, Karen, Bruce, Pauline and Andrew

Header Photo by engin akyurt on Unsplash

Modelling Government in Scotland on Wikidata

This was one of three projects which were worked on during CTC24 – Open In Practice. We asked Jan Ainali, who led the project, to explain it for those who were not present at the weekend event.

Why did we run this project? 

On Wikidata, there is a WikiProject for getting all the government agencies of all levels and the whole world properly modeled. Since it is a huge project, every way to try to break it down to bite size pieces are necessary. Work on the UK was already started, so it made a lot of sense trying to complete Scotland.

Work done before CTC24

In October during Scottish Open Data Unconference 2021 we got started on this task. We found some good sources and made fine progress, completing several categories of agencies. By completing here, I mean that we made sure there were items in Wikidata representing the agencies and that they were well enough modeled so that we could query for them. But we weren’t done, and some of the trickiest parts remained. 

What we achieved at CTC24, what impact we hope it will have

With a joint effort, we managed to sort out how the judicial system was organized, which was something that remained unclear since the last session. Most time was spent in researching to understand it, and when that was done, it was fairly straightforward to create items for the courts that were missing and to model the others in a way that made it possible to query for them. We also managed to sort out some other small tasks from the last event, and finally we could produce one huge query to get all Scottish agencies at once:  https://w.wiki/4TpN

Now we think this is the most complete and up-to-date list of Scottish agencies. If we are wrong about that, we would love feedback so that we can improve it.

What next – how can people get involved?

A few things are happening right now. First, we are importing agencies into the Govdirectory platform that is a more user-friendly view of Wikidata. We have already imported the local authorities, NHS boards and Health and Social Care Partnerships. You can find that data here: https://www.govdirectory.org/united-kingdom/

Since the large query was a bit messy, we will also try to improve the modeling in Wikidata. You are more than welcome to help with this. This will make queries for everybody simpler, and we will continuously be importing more agencies to the platform as we get done.

You can also help by adding contact points like email, official websites, social media accounts to the Wikidata items. You can either use the Wikidata button on the Govdirectory website, or you can go to the WikiProject page on Wikidata and run some queries there to find items to improve. https://www.wikidata.org/wiki/Wikidata:WikiProject_Govdirectory/United_Kingdom

Header Image by Jan Ainali, CC0, via Wikimedia Commons

Make a difference with us

At Code the City we believe that the right people, with the right skills and tools, can do great things. We believe that we can use technology and data to solve many civic challenges. Those beliefs are as applicable now as was when we started seven years ago. And our volunteers who come to our events time and again agree. They know that sharing their skills and knowledge with others in small teams, over a weekend, working on a focussed and achievable project, is a satisfying experience which leaves them with a sense of achievement. It also introduces them to working in teams and in an agile way: short sprints of work and pauses for review. 

“The power of one, if fearless and focussed, is formidable, but the power of many working together is better” –

Gloria Macapagal Arroyo

In the last seven years we’ve tackled many topics – and worked with multiple partner organisations in the public and private sector to solve their challenges – and to identify opportunities to use data and technology to improve how they deliver their services.

Throughout that period we’ve had some central principles that we’ve adopted which still hold true: 

  • Data, where appropriate, should be open and licensed for reuse
  • Software should be developed as open source – where the code can be inspected, and improved on by anyone, and reusable openly by others 
  • Information, images and other content should be as openly licensed as possible to encourage re-use and creativity
  • Where appropriate stable platforms exist (such as WIkidata, Open Streetmap, Github,or Wiki Commons) we should use those
  • People working in small teams and in short sprints of activity can achieve an enormous amount over a weekend

Last week at Open UK’s COP26 event “Open Technology for Sustainability”, which our co-founder and trustee Ian Watt attended, those same principles that inspired our creation, and inform our continuing work, were echoed time and again by speakers. And at the evening awards dinner we were runners-up to the the wonderful Open Knowledge Foundation, in the Data category. This further validates our belief in our approach. 

CTC’s Runner-up trophy for data in Open UK’s 2021 awards

More recently we’ve been concentrating even harder on improving open data in Scotland and the UK – but not to the exclusion of other projects. In addition to several history and heritage projects which have seen large amounts of open data created and published, we’ve had projects such as Open Wastemap which was built almost entirely over two CTC weekend and uses community-sourced data in Wikidata and OpenStreetmap to power this really useful tool to find local recycling facilities. 

Our next event CTC24 – Open In Practice is taking place in just over a week. It is the perfect introduction to what we do and to becoming involved. We already have a list of potential projects that attendees, new and experienced, can get involved in. Some of these are local in scale and some national. All need a blend of skills from attendees. You don’t need to be either a coder or data expert to participate. You can sign up directly here or from the event link above. 

No excuses: be part of the group that does the good things – or stand by and watch while we do!

How Our Events Deliver an Unforgettable Experience

One of the biggest strengths of the events we run at CTC is the wide variety of projects we undertake, with each one being engaging in a completely different way. In our last blog post we looked at the ‘History + Data = Innovation’ and ‘History and Culture’ events which highlighted interesting records from the 19th and 20th century and which generate new open data from historic sources. The Event we’re looking at today however, ‘Archeology Meets Data Science’ uncovers an even older part of Aberdeen’s History.

To give some context, an excavation of St Nicholas Kirk from 2006-2007 uncovered more than a thousand human remains and artefacts, as well as parts of the building dated at over a thousand years old. In 2018 we were granted the opportunity to work with data taken from the excavation as well as work with some of the human remains themselves – certainly not the usual kind of work you’d expect to carry out at a hack event! Six teams were formed to work on various aspects of the project such as working through the original data and compiling supplementary info in an extensive Q&A document.

Image

The team which mostly worked with the Skeletons used a technique called Photogrammetry – this involves taking photos from many angles which with the right software, can be used to create a 3d model. 

While some experts were involved with the project many of the team had never done something like this, so there was a lot of trial and error with finding the right software. Eventually their perseverance paid off however, and the result is a great looking archive of the scanned remains freely available online. 

As well as this we also had a team of 3d modellers use Unity to recreate the burial site as well as where the skeletons were located. There’s even Virtual Reality support if you own an Oculus Go, and the burial can be accessed here. 

Ali Cameron was one of the experts involved in the project, having been involved in the field for more than 40 years, and was impressed with the work we carried out. We asked about her thoughts regarding the project, she had the following to say:  

Some of the quieter students really came out of their shells, we all got a chance to meet the coders and I have kept up with a couple of them to discuss other projects. The coders really enjoyed the archaeology side and I chatted with them all about the aspect they were working on which is quite different from a lot of the programming they had done before. The Event was extremely successful and very fascinating.

Overall the project turned out to be quite an unforgettable experience for those involved. It challenged our team and volunteers in an interesting way, and was a unique chance to interact with some of the oldest relics in Aberdeen’s history. It also highlights our goal of making data accessible very well, as we took a fascinating discovery and allowed its contents to be made freely available online thanks to our 3d modelling endeavors.We have a more in-depth post regarding the project for those wanting to find out more.

If the work we carry out interests you then our next event, ‘CTC24 – Open in Practice’, is only a few weeks away, with more information and booking links here.