It’s purpose is to gather data on Aberdeen-built ships, with the permission of the site’s owners, and to push that refined bulk data, with added structure, onto Wikidata as open data, with links back to the Aberdeen Ships site through using a new identifier.
By adding the data for the Aberdeen Built Ships to Wikidata we will be able to do several things including
Create a timeline of ship building
Create maps, charts and graphs of the data (e.g. showing the change in sizes and types of ships over time
Show the relative activity of the many shipbuilders and how that changed
Agree a core set of data for each ship that will parsed from ships.json to be added to Wikidata – e.g. name, year, builder, tonnage, length etc
Create a script to output text that can be dropped into a CSV or other file to be used by QuickStatements (assuming that to be the right tool) for bulk input ensuring links for shipbuilder IDs and ABS identifiers are used.
We will also be looking to get pictures of the ships published onto Wiki Commons with permissive licences, link these to the Wiki Data and increase and improve the number of Wikipedia articles on Aberdeen Ships in the longer-term.
On Saturday 14th December 2019 we ran a one-day mini hack event. The idea behind it was for people to come along for a day to work on their side projects and, if they needed support, attempt to persuade others to assist them.
That’s what I did with my Aberdeen Plaques project: something I’d had on the back burner for more than a year.
Why do it?
The commemorative plaques which are dotted around the city are a perfect candidate for open data. They have a subject, usually some dates, are located somewhere, and are of different types etc. Making that all available as open data would open up a whole range of possibilities.
If we captured all of that well then we could do analysis on the data (ratio of women to men, most represented professions), create walking routes (maybe one for the arts, one for the sciences and so on), create timelines to see what periods are more represented.
Having recently trained as a WikiMedia UK trainer – and having experimented with some of the tools (Wiki Commons, Wiki Data, Wikipedia, Histropedia) I was convinced that these were the right way to go.
So, in advance of the hack day I’d done a bit of prep in the two weeks running up to the day iteself.
I’d created a spreadheet which recorded the
* subject (person or ‘thing’)
* Gender if known
* the link to the now-retired city council plaques system (hidden from public view)
* The location if known
* The geo coordinates (to be determined)
* Whether the subject had a Wikipedia page (tbd)
* Whether there was an image of the plaque on Wiki Commons (tbd)
* Whether the subject of the plaque was represented on Wiki Data (tbd)
* Any identifiers on Open Plaques (tbd)
* Any external links (eg to Flickr for photos)
I’d then populated some of the data (eg whether there were images of the plaque on Wiki Commons) as well as some other bits. But most cells were blank.
As a keen walker and photographer I had also photographed and uploaded seventeen plaque images to Wiki Commons in the lead up, so that we would have some images to work with.
How to use our time most effectively on the day?
Our aim for the day was then to find out what data / info / images existed, fill in the gaps, and explore how to use WikiData to store and retrieve data, and how we could potentially create maps, timelines and similiar new products.
What we did on the day
At the start of the event we pitched our project ideas, and I managed to persude five others (Angela, Mike, Stephen, James and Steve) to join me in working on the plaques project.
Angela and Mike, and later Angela and Stephen would go out and take photographs. Steve, James and I would work on the data capture, completing research on what existed, creating new entries for the data on Wiki Data, and testing queries on the Wiki Data query service.
How we did it
We used the spreadsheet that I had set up to capture all of the data we’d gathered – and as it eveolved it would show progress as well as what was still lacking. We had no expectations that we would do it all on the day, but we could pick away at it in future weeks and months.
In the run-up to the event I’d discovered The Pingus’ album of plaques photographs on Flickr. Sadly these had not been published with a licence that would allow us to use them. I’d sent a request, a few days before CTC18, for them to change the licence for the Aberdeen plaques pictures to a CC-SA one. This would have allowed our republishing on Wiki Commons. Sadly it didn’t elicit a response. But the album did show that there were many more plaques than the old ACC system listed. And it was possible to get co-ordinates from them. So the number of plaques to deal with kept growing.
During the day James filled in loads of gaps in which subjects were on Wikipedia and which on Wikidata.
Steve and I experimented with capturing and querying the data. Structuring that in a way that aids recall through Wiki Data Query Service was an interative process. Firstly I tried adding a statement ‘commomorative plaque image’ (P1801) into the wikidata record for the subject as you can see in this first example https://www.wikidata.org/wiki/Q2095630. But that limited what we could do.
So, we discovered that we could create a new object which was an instance of commemorative plaque. Our first attempt was https://www.wikidata.org/wiki/Q78438703 and we evolved what we captured there – adding statement, and Steve discovered the ‘openPlaques plaque ID'(P1893). Incidentally we also tried ‘openplaques Subject ID’ (P1430) but adding that to the plaque object throws an error. The latter should be added to the person record not the plaque.
At the end of CTC18
We ended the day with
138 plaques listed.
57 sets of co-ordinates identified
68 Wikipedia articles identified as matching plaque subjects (and eleven plaques subjects who had NO wikipedia page)
36 Images in WikiCommons
77 WikiData entries for the subject of the plaques (existing or created)
11 new wikidata entries for the plaques themselves
This was a great leap forward in one day and would pave the way for future work.
Since CTC18 ended, I’ve got firmly stuck into this project over the xmas break. Over the last three weeks I have now photographed over a hundred plaques (plenty of walking) and have created wikidata entries for most plaques and also their subjects in wikidata.
I’ll cover all of that, and how we can now use the data in part two, coming soon.
During the course of Code The City 17: Make Aberdeen Better this weekend we made a startling discovery. It is easier to recycle your old fridge-freezer than to get data and content for re-use from Scottish public sector websites. As a consequence, innovating new solutions to common problems and helping make things easier for citizens is made immeasurably more difficult.
One of the event’s challenges posed was “How do we easily help citizens to find where to recycle item ‘x’ in the most convenient fashion. That was quickly broadened out to ‘dispose of an item” since not everything can be recycled – some might be better reused, and others treated as waste, if it can’t be reused or recycled. With limited kerbside collections, getting rid of domestic items mainly involves taking them somewhere – but where?
With climate change, and the environment on most people’s minds at the moment, and legislative and financial pressures on local authorities to put less to landfill, surely it is in everyone’s interest to make it work as well as it can.
To test how to help people to help themselves by giving advice and guidance, we came up with a list of 12 items to test this on – including a fridge, a phone charger, a glass bottle, and tetra pack carton. On the face of it this should be simple, and probably has been solved already.
The Github Repo
All of Code The City hack weekend projects are based on open data and open source code. We use Github to share that code – and any other digital artefacts created as part of the project. All of this one’s outputs can be found (and shared openly) here.
That was where we started: looking to see if the problem has already been solved. There is no point in reinventing the wheel.
We looked for two things – apps for mobile phones, and websites with appropriate guidance.
Aberdeen specific information?
Since we were at an event in Aberdeen we first looked at Aberdeen City Council’s website. What could we find out there?
Not much as it turned out – and certainly not anything useful in an easy-to-use fashion. On the front page there was an icon and group of suggested services for Bins and recycling; none of which were what we were looking for.
Typing recycling into the search box (and note we didn’t at this stage know if our hypothetical item could be recycled) returned the first 15 of 33 results. As shown below.
The results were a strangely unordered list – neither sorted alphabetically nor by obvious themes. So relevant items could be on page 3 of the results. Who wants to read policies if they are trying to dispose of a sofa? Why are two of (we later discovered) five recycling centres shown but three others not? Why would I as a citizen want to find out about trade waste when I just want to get rid of a dodgy phone charger?
Why is there a link to all recycling points (smaller facilities in supermarket carparks or such like, with limited acceptance of items), but apparently not to all centres which cover much more items? Actually there is a link ‘Find Your Nearest Recycling Centre’ (but not your nearest recycling point which are much more numerous). This takes you a map and tabular list of centres and what they accept. And it is easy to miss the search box between the two. No such facility exists for the recycling points.
Perhaps there is open data on the ACC Data portal that we could re-purpose – allowing us to build our own solution? Sadly not – the portal has had the same five data sets for almost two years, and every one of those has a broken link to the WMSes.
If we were in Dundee we could download and use freely their recycling centre data. But not in Aberdeen.
Apps to the rescue?
There are some apps and services that do most of what we are trying to do. For example iRecycle – Iphone and Android is a nice app for Android and iOS that would work were it not for US locations only.
We couldn’t find something for Scotland that worked as an App.
Other sources of information?
Since we drew a blank as far as both Aberdeen City Council and any useable apps, we widened our search.
Recycle For Scotland
The website Recycle For Scotland (RFS) is, on the face of it a useful means to identify what to do with a piece of domestic waste. Oddly, there appears not to be any link to it that we could find from any of the ACC recycling pages.
BUT …… it doesn’t work as well as it could and the content, and data behind it have no clear licence to permit reuse.
The Issues with RFS
Searching the site, or navigating by the menus, for Electrical Items results in a page that is headed “This content was archived on 13th August 2018” – hardly inspiring confidence. No alternative page appears to exist and this page is the one turned up in navigation on the site.
Searching for what to do with batteries in Aberdeen results in a list of shops at least one of which closed down about 18 months ago. Entering a search means entering your location manually – every time you search! This quickly becomes wearing.
While the air of neglect is strong, the site is at least useful compared to the ACC website. But it doesn’t do what we want. Perhaps we could re-use some of the content? No – there is no clear licence regarding reuse of the website’s content.
ZWS are publicly funded by the Scottish Government and the European Regional Development Fund – all public money.
Public funding should equal open licences
We argue that any website operated by a government agency, or department, or NDPB, should automatically be licensed under the Open Government Licence (OGL). And any data behind that site should be licensed as Open Data.
Changing the licensing of Recycle For Scotland website, making its code open source, and making its data open would have many benefits.
its functionality could be improved on by anyone
the data could be repurposed in new applications
errors could be corrected by a larger group than a single company maintaining it.
Where did this leave us?
Having failed to identify an app that worked for Scotland, nor interactive guidance on the ACC website, we tried the patchy and, on the face of it, unreliable RFS site. We’d turned to the data and whether we could construct something useable from open data and repurposed, fixed, content over the weekend – this is a hack event after all.
But in this we were defeated – data is wrapped up in web pages: formatted for human readability, not reuse in new apps.
Websites which were set up to encourage re-use and recycling ironically prohibit that as far as their content and data is concerned, and deliberately stifle innovation.
Public funding from the City Council, the Scottish Government and the European Regional Development Fund is used to fund sites which you have paid but elements of which you cannot reuse yourself.
At a time of climate crisis, which the Scottish Government has announced is a priority action, it can’t be right that not only is it difficult to find ways to divert domestic items from landfill, but also that these Government-funded websites have deliberate measures in place to stop us innovating in order to make access to reuse and recycle easier!
Hopefully politicians, ministers and councillors will read this (please draw it to their attention) and wake up to the fact that Scotland deserves, and needs, better than this.
Only by having an Open Data by default policy for the whole of the Scottish Public Sector, and an open government licence on all websites can we fix these problems through innovation.
After all if the non-functioning Northern Ireland Assembly can come up with an open data strategy that commits the region to open data by default, why on earth can’t Scotland?
“Northern Ireland public sector data is open by default. Open by default is the first guiding principle that will facilitate and accelerate Open Data publication.”
[Edit – Added 12-Nov-2019]
If you are interested to read more about the poor state of Scottish Open Data you might be interested in this post I wrote in February 2019 which also contains links to other posts on the subject:
Last weekend we hosted the second Aberdeen Air Quality hack weekend in recent months. Coming out it there are a number of tasks which we need to work on next. While some of these fall to the community to deliver, there are also significant opportunities for us to work with partners.
While the Air Aberdeen website is better, we still need to apply the styling that was created at the weekend.
We’ve established that the DHT022 chips which we use in the standard Luftdaten device model have challenges in working in our maritime climate. They get saturated and stop reporting meaningful values. There is a fix which is to use BME380 chips in their place. These will continue to give humidity and temperature readings, plus pressure, but due to the different technology used will handle the humidity better. Knowing local humidity is important (see weather data below). So, we need to adapt the design of all new devices to use these chips, and retrofit the existing devices with the new chips.
Placement of new devices
We launched in February with a target of 50 sensors by the end of June and 100 by the end of the year. So far attendees have built 55 devices of which 34 are currently, or have recently been, live. That leaves 21 in people’s hands that are still to be registered and turned on. We’re offering help to those hosts to make them live.
Further, with the generous sponsorship of Converged, Codify, and now IFB we will shortly build 30 more devices, and that will take us to a total of 85. We’ve had an approach by a local company who may be able to sponsor another 40. So, it looks like we will soon exceed the 100 target. Where do we locate these new ones? We need to have a plan to strategically place those around the city where they would be most useful which is where the map, above, comes in.
Community plus council?
We really want to work with the local authority on several aspects of the project. It’s not them versus us. We all gain by working together. There are several areas that we could collaborate on, in addition to the strategic placement of future devices.
For example, we’ve been in discussions with the local authority’s education service with a view to siting a box on every one of the 60 schools in the city. That would take us to about 185 devices – far in excess of the target. Doing that needs funding, and while the technology challenge to get them on the network is trivial, ensuring that the devices survive on the exterior of the buildings might be a challenge.
Also, we’ve asked but had no response to our request to co-locate one of our devices on a roadside monitoring station which would allow us to check the correlation between the outputs of the two. We need to pursue that again.
Comparing our data suggests that we can more than fill in gaps in the local council’s data. The map of the central part of Aberdeen in the image above, shows all of the six official sensors (green) and 12 of the 24 community sensors that we have in the city (in red). You can also see great gaps where there are no sensors which again shows the need for strategic placement of the new ones.
We’ve calculated that with a hundred sensors we’d have 84,096,000 data observations per year for the city, all as open data. The local authority, with six sensors each publishing three items of data hourly, have 157,680 readings per annum – which is 0.18% of the community readings (and if we reach 185 devices then ACC’s data is about 0.10% or 1/1000th of the community data) and the latter of course, besides being properly open-licensed, has much greater granularity and geographic spread.
We need to ensure that we gather historic and new weather data and use that to check if adjustments are needed to PM values. Given that the one-person team who was going to work on this at CTC16 disappeared, we need to first set up that weather data gathering, then apply some algorithms to adjust the data when needed, then make that data available.
Engagement with Academia
We need to get the two local universities aboard, particularly on the data science work. We have some academics and post-grads who attend our events, but how do we get the data used in classes and projects? How do we attract more students to work with us? And , again we need to get schools to only hosting the devices but the pupils using the data to understand their local environment?
The cool stuff
Finally, we when we have the data collected, cleaned, and curated, and APIs in place (from the green up through orange to red layers below) we can start to build some cool things (the blue layers).
These might include, but are not limited to:
data science-driven predictive models of forecast AQ in local areas,
public health alerts,
mobile apps to guide you where it is safe to walk, cycle, jog or suggest cleaner routes to school for children,
logging AQ over time and measuring changes,
correlating local AQ with admissions to hospital of cases of COPD and other health conditions
inform debate and the formulation of local government strategy and policy.
As we saw at CTC16, we could also provide the basis for people to innovate using the data. One great example was the hacked LED table-top lamp which changes colour depending on the AQ outside. Others want to develop personalised dashboards.
Tourism is vital to the local economy. While loads of tourists pass through Aberdeen we could do so much more to make it a destination of choice.
The featured image above is how we used to attract tourists to Aberdeen. How should we do it now? What role does design, marketing, technology or data play in new interactions with tourists?
Who should attend?
Anyone – despite our name, coding is a small bit of what we do.
Of course, coders, data wranglers, designers and other techies are important to a hack weekend.
We’d be delighted to see you if:
You work in the tourism sector, operating attractions, providing accomodation or other services
You have identified a problem with service delivery, or see an opportunity to do things better!
You have an interesting in service design
You work or study in the creative industries
You have experience as a tourist in Aberdeen or anywhere else
You are someone who wants to do more with data but isn’t sure where to start
You are a student (of any discipline)
You are someone who wants to improve the local area, or use digital and skills to improve local services
You are from the third sector or local government
You work with mapping, GIS, or location data
You are curious about learning new techniques and skills to use in your day job, and finally
And, of course, if you are a developer, designer, UX expert, data wrangler, coders, or service designer
So it’s a Toursim hack?
Yes – we’ll be identifying opportunities and barriers to making Aberdeen City and Shire a destination of choice for tourists, creating projects, teams and prototypes to address those. Some of those will turn into coding projects – and some will all be about research, service design and paper prototyping.
The event will run Saturday 9.30am to about 5pm, then Sunday 9.30 to about 4.30pm.
What happens over the weekend?
* identification of opportunities and barriers
* ideation to address those
* creation of project teams to work on those
* agile prototyping of solutions, so that by close of play Sunday we will have demonstrable solutions which could be developed into real worls products or services.
But, a ticket will cost £5 (*)
What do you mean, it’s not free? CTC is usually free!
At CTC 11 in December we broke with a tradition established over the previous 10 events and charged a small amount to attend. The reasone was that in a couple of recent CTC events we had higher than normal numbers of people booking free tickets and not showing up. That meant we over-catered, and despite our best efforts we had left-over food, which is a bad thing and wastes money too.
So, we attached a monetary value to the ticket (* backed up by our promise that those attending would get their money back when they showed up). And it worked. The bookings didn’t go down. Fewer people dropped out. We didn’t waste food and we banked the money that went unclaimed. A few generous individuals recognising our new charity status even refused their money back, which was nice.
We’ll be at the Sir Ian Wood Building at Robert Gordon University’s Garthdee Campus. You can get a No 1 bus from King Street / Union Street which will take you into the campus and drop you at the door. Or you can get a No. 2 bus which will drop you at the gates to the campus.
If you must drive, then parking is free and open at weekends on campus.
Robert Gordon University will be sponsoring this event. We could do with another sponosor or two to make sure we cover all costs. If you would like to sponsor it, get in touch with @codethecity on Twitter.