The Od-Bods project: update from CTC24

Why did we run this project? 

Theoretically, with the 2015 Scottish Government commitment to data being “open by default”, we should have universal publication of appropriate data as open data. In reality Scotland is very poorly served with Open Data. Few local authorities publish any, and those who do have little consistency. Beyond councils the picture is, if anything, even worse. Finding data is all but impossible. We set out to make data more findable, identify who is making data available and, perhaps as importantly, those who are not. We began with local government.

This work is a starting point, not an end point. 

Work done before CTC24

  • We wrote this blog post about this project to accompany the work done at CTC23, the forerunner to this event. 

What we achieved at CTC24, what impact we hope it will have

What challenges we have faced/are currently facing?

  • As always, lack of (good) engagement  with public sector.
  • There is no standardisation in how and where local government published its data.
  • Gathering data and cleaning it to output in a presentable format is currently a manual and laborious process.

What next – how can people get involved?

  • We have a page explaining the project, what our objectives are and what the plan is: https://opendata.scot/about/
  • There’s a big list of GitHub issues to be worked on here: https://github.com/OpenDataScotland/the_od_bods/issues 
  • The current milestone for Q1 2022 is to improve our data that we have gathered so far:
    • Fix known bugs with API calls
    • Tidy up inconsistent dataset tags
    • Identify and locate any missing data from the 32 local authorities which we haven’t found yet
    • Add more data features/metadata if possible

Join us at CTC25 to work on the project issues and work toward our next milestone! 

Modelling Government in Scotland on Wikidata

This was one of three projects which were worked on during CTC24 – Open In Practice. We asked Jan Ainali, who led the project, to explain it for those who were not present at the weekend event.

Why did we run this project? 

On Wikidata, there is a WikiProject for getting all the government agencies of all levels and the whole world properly modeled. Since it is a huge project, every way to try to break it down to bite size pieces are necessary. Work on the UK was already started, so it made a lot of sense trying to complete Scotland.

Work done before CTC24

In October during Scottish Open Data Unconference 2021 we got started on this task. We found some good sources and made fine progress, completing several categories of agencies. By completing here, I mean that we made sure there were items in Wikidata representing the agencies and that they were well enough modeled so that we could query for them. But we weren’t done, and some of the trickiest parts remained. 

What we achieved at CTC24, what impact we hope it will have

With a joint effort, we managed to sort out how the judicial system was organized, which was something that remained unclear since the last session. Most time was spent in researching to understand it, and when that was done, it was fairly straightforward to create items for the courts that were missing and to model the others in a way that made it possible to query for them. We also managed to sort out some other small tasks from the last event, and finally we could produce one huge query to get all Scottish agencies at once:  https://w.wiki/4TpN

Now we think this is the most complete and up-to-date list of Scottish agencies. If we are wrong about that, we would love feedback so that we can improve it.

What next – how can people get involved?

A few things are happening right now. First, we are importing agencies into the Govdirectory platform that is a more user-friendly view of Wikidata. We have already imported the local authorities, NHS boards and Health and Social Care Partnerships. You can find that data here: https://www.govdirectory.org/united-kingdom/

Since the large query was a bit messy, we will also try to improve the modeling in Wikidata. You are more than welcome to help with this. This will make queries for everybody simpler, and we will continuously be importing more agencies to the platform as we get done.

You can also help by adding contact points like email, official websites, social media accounts to the Wikidata items. You can either use the Wikidata button on the Govdirectory website, or you can go to the WikiProject page on Wikidata and run some queries there to find items to improve. https://www.wikidata.org/wiki/Wikidata:WikiProject_Govdirectory/United_Kingdom

Header Image by Jan Ainali, CC0, via Wikimedia Commons

Make a difference with us

At Code the City we believe that the right people, with the right skills and tools, can do great things. We believe that we can use technology and data to solve many civic challenges. Those beliefs are as applicable now as was when we started seven years ago. And our volunteers who come to our events time and again agree. They know that sharing their skills and knowledge with others in small teams, over a weekend, working on a focussed and achievable project, is a satisfying experience which leaves them with a sense of achievement. It also introduces them to working in teams and in an agile way: short sprints of work and pauses for review. 

“The power of one, if fearless and focussed, is formidable, but the power of many working together is better” –

Gloria Macapagal Arroyo

In the last seven years we’ve tackled many topics – and worked with multiple partner organisations in the public and private sector to solve their challenges – and to identify opportunities to use data and technology to improve how they deliver their services.

Throughout that period we’ve had some central principles that we’ve adopted which still hold true: 

  • Data, where appropriate, should be open and licensed for reuse
  • Software should be developed as open source – where the code can be inspected, and improved on by anyone, and reusable openly by others 
  • Information, images and other content should be as openly licensed as possible to encourage re-use and creativity
  • Where appropriate stable platforms exist (such as WIkidata, Open Streetmap, Github,or Wiki Commons) we should use those
  • People working in small teams and in short sprints of activity can achieve an enormous amount over a weekend

Last week at Open UK’s COP26 event “Open Technology for Sustainability”, which our co-founder and trustee Ian Watt attended, those same principles that inspired our creation, and inform our continuing work, were echoed time and again by speakers. And at the evening awards dinner we were runners-up to the the wonderful Open Knowledge Foundation, in the Data category. This further validates our belief in our approach. 

CTC’s Runner-up trophy for data in Open UK’s 2021 awards

More recently we’ve been concentrating even harder on improving open data in Scotland and the UK – but not to the exclusion of other projects. In addition to several history and heritage projects which have seen large amounts of open data created and published, we’ve had projects such as Open Wastemap which was built almost entirely over two CTC weekend and uses community-sourced data in Wikidata and OpenStreetmap to power this really useful tool to find local recycling facilities. 

Our next event CTC24 – Open In Practice is taking place in just over a week. It is the perfect introduction to what we do and to becoming involved. We already have a list of potential projects that attendees, new and experienced, can get involved in. Some of these are local in scale and some national. All need a blend of skills from attendees. You don’t need to be either a coder or data expert to participate. You can sign up directly here or from the event link above. 

No excuses: be part of the group that does the good things – or stand by and watch while we do!

How Our Events Deliver an Unforgettable Experience

One of the biggest strengths of the events we run at CTC is the wide variety of projects we undertake, with each one being engaging in a completely different way. In our last blog post we looked at the ‘History + Data = Innovation’ and ‘History and Culture’ events which highlighted interesting records from the 19th and 20th century and which generate new open data from historic sources. The Event we’re looking at today however, ‘Archeology Meets Data Science’ uncovers an even older part of Aberdeen’s History.

To give some context, an excavation of St Nicholas Kirk from 2006-2007 uncovered more than a thousand human remains and artefacts, as well as parts of the building dated at over a thousand years old. In 2018 we were granted the opportunity to work with data taken from the excavation as well as work with some of the human remains themselves – certainly not the usual kind of work you’d expect to carry out at a hack event! Six teams were formed to work on various aspects of the project such as working through the original data and compiling supplementary info in an extensive Q&A document.

Image

The team which mostly worked with the Skeletons used a technique called Photogrammetry – this involves taking photos from many angles which with the right software, can be used to create a 3d model. 

While some experts were involved with the project many of the team had never done something like this, so there was a lot of trial and error with finding the right software. Eventually their perseverance paid off however, and the result is a great looking archive of the scanned remains freely available online

As well as this we also had a team of 3d modellers use Unity to recreate the burial site as well as where the skeletons were located. There’s even Virtual Reality support if you own an Oculus Go, and the burial can be accessed here

Ali Cameron was one of the experts involved in the project, having been involved in the field for more than 40 years, and was impressed with the work we carried out. We asked about her thoughts regarding the project, she had the following to say:  

Some of the quieter students really came out of their shells, we all got a chance to meet the coders and I have kept up with a couple of them to discuss other projects. The coders really enjoyed the archaeology side and I chatted with them all about the aspect they were working on which is quite different from a lot of the programming they had done before. The Event was extremely successful and very fascinating.

Overall the project turned out to be quite an unforgettable experience for those involved. It challenged our team and volunteers in an interesting way, and was a unique chance to interact with some of the oldest relics in Aberdeen’s history. It also highlights our goal of making data accessible very well, as we took a fascinating discovery and allowed its contents to be made freely available online thanks to our 3d modelling endeavors.We have a more in-depth post regarding the project for those wanting to find out more.

If the work we carry out interests you then our next event, ‘CTC24 – Open in Practice’, is only a few weeks away, with more information and booking links here.

You Don’t Need to Know How to Code to Enjoy Our Events

While the projects we carry out may seem a bit daunting to newcomers, everyone involved in our events carries out a vital role which is highlighted by some past attendee experiences. 

As part of our commitments to making Data accessible for all we were looking to digitally archive many 19th and 20th century records that were only available physically or were missing information online. This was the main focus of our 19th and 20th Events known as ‘History + Data = Innovation’ and ‘History and Culture’ respectively. These primarily involved uploading data on a multitude of subjects to Wikidata such as Listed buildings, convict registers and March Stones, which signified the boundary of crofts in Aberdeen primarily in the 16th century.

Heather Black initially found out about the project through the Aberdeen memories Facebook page and got involved transcribing part of the Aberdeen harbour logbooks; specifically the names, registered countries, the name of the captain and what cargo was being shipped.

A page of the harbour arrivals register for 11 Nov 1920.

Having a keen interest in local history the project immediately drew her attention, and when asked about her thoughts on the event, she stated that “It was a good distraction while being on furlough from my work at the time, and very interesting too”, and is looking to participate in future events that catch her interest. 

Sheila Watt was similarly involved in this project, and despite not having too much interest initially ended up having a great experience. Her daughter joined Code the City after looking for a volunteering project to take part in for a Duke of Edinburgh Award last year. Eventually Sheila joined as well after hearing about the Aberdeen Harbour Arrivals project and due to her interest in local history ended up  greatly enjoying her experience, describing it as the perfect project to keep her interested during the first lockdown last year.

Taking on the role of a volunteer transcriber, she enjoyed her experience so much that she took part again for the Returned Prisoner Project and similarly had a great time in the same role. She now regularly checks on our projects through the slack channel and will hopefully be involved in future projects. 

Despite some participants being apprehensive about the tech-side of things, there are lots of ways to contribute to each event, and you’ll definitely find something that plays to your strengths and skills. If you’re interested in any of the event topics you’ll find the work engaging too, and get to talk to some friendly like-minded people.

Check this post to learn more about the Harbour archival process as well as the wider project. 

Our next event is CTC24 Open in Practice where we will have several projects that will appeal to non-coders, offering the opportunity to gather more data and open it up for public benefit.