Up-coming events

2019 got off to a roaring start with the first data meetup of the year, and one of our biggest attendances! There is lots of interest in CTC15 Air Quality event in February. 70% of sensor builder tickets have gone! If you are thinking about going don’t delay!

  • 5th February 2019 – Data Meetup  – Anatomy of a Data Science Project // Uplift Modelling: Spend Less & Sell More
  • CTC15 Air Quality – 16th and 17th Feb 2019 Get involved in an Air Quality project covering the whole of Aberdeen.  Tickets now on sale. 
  • 12 March 2019 – Data Meetup – Topics to be announced soon.
  • 16-17 March 2019 – CTC Hack weekend (to be confirmed)
  • 2nd April 2019 – Data Meetup. Our first anniversary!

To get advanced notice of our events, and make sure of a place, why not sign-up for our monthly, spam-free, mailing list?

Aberdeen Air Quality

How much do you care about the quality of the air you breathe as you walk to work or university, take the kids to school, cycle or jog, or open your bedroom window?

How good is the air you are breathing? How do you know? What are the levels of particulates (PM2.5 or PM10) and why is this important?

pm25_comparison
pm25_comparison

When do these levels go up or down? What does that mean?

Who warns you? Where do they get their data, and how good is it?

Where do you get information, or alerts that you can trust?

We aim to sort this in Aberdeen

Partnering with community groups, Aberdeen University and 57 North Hacklab, we are working on a longterm project to build and deploy community-built, and hosted, sensors for PM2.5 and PM10. We aim to have fifty of these in place in the next few months, across Aberdeen. You can see some early ones in place and generating data here.

The first significant milestone of this will be the community workshop we are holding on 16-17 February 2019. If you want to be part of it, you can get a ticket here. But, be quick; they are going quickly.

Weekend activities

There are loads of things you can do if you attend.

Sensor Building

For a small cost, you can come along and build your own sensor with someone to help you, and take it home to plug into your home wifi. It will then contribute data for your part of the city.

But we will be doing much more than that.

Working with the data

If you have experience in data science or data analysis, or if you want to work with those who do, there are loads of options to work with the data from existing and future sensors.

These include

  • Allow historical reading to be analysed against the official government sensors for comparison
  • Use the data; wind speed, humidity… to build live maps of readings to identify sources of emissions.
  • Compensate readings from sensors against factors which affect pollution levels to attempt to understand the emissions of pollutants in a given area.
  • Build predictive models of future pollution
  • Fix a minor issue with the existing data Collected Data (see https://github.com/opendata-stuttgart/madavi-api/issues/8 )
  • Build an API for the access of the Luftdaten sensor data to allow querying of the sensor data

Software development

If you are a software developer or studying to be one, you could

  • Create alerts systems to warn of anticipated spikes in pollutants, perhaps using Twitter, or email.
  • Add to the code for the Luftdaten sensors to allow connection over LoRaWAN interface.
  • Create LoRaWAN server code to allow sensors to feed up to the Luftdaten website.
  • Security testing of the IoT Code used by the Luftdaten sensors.

Community Groups / Educators / Activists / Journalists

You don’t have to be a techie! If you are a concerned citizen, and community activist, a teacher, or a journalist there is so much you could do. For example:

  • How can you understand the data?
  • Identify how this could assist with local issues, campaigns, educational activities.
  • Help us capture the weekend by blogging, or creating digital content

Even if you just want to be part of the buzz and keep the coffees and teas flowing, that is a great contribution.

See you there!

Ian, Bruce, Andrew and Steve

 

Header image by Jaroslav Devia on Unsplash

2018 – A year in review

2018 has been a really busy year for us. Here are all the things that we delivered.

Open Data Camp

We hosted UK Open Data Camp’s first ever visit North of the border in November. Over a hundred people travelled to Aberdeen for two days of unconferencing where there were 44 sessions run on a variety of data-related topics. Some people went for an Aberdeen version of the Joy Diversion walk around old Aberdeen, and others discovered the pleasure of logging Open Benches. The feedback was overwhelmingly positive and there were loads of write-ups.

Code the City Hack Weekends

We had two great Code The City Events: CTC13 – Hacking our Relationship with Alcohol, and CTC14 – Archaeology. Both were well attended and produced some very interesting results. The first saw us tackling some interesting real-world problems, helping people to overcome problems, and build a machine learning model to predict whether a beer can design would be more likely to be perceived as alcoholic or not. A report of the weekend is being written as an academic paper for a forthcoming health conference!

 

The second weekend saw us scanning and creating 3D renders of six real skeletons with mobile phones. We also began to create a 3D model of the church in which the dig took place, and generate data from written logs to populate that.

Well done to all who participated. We got some great feedback on each event.

Data Meetups

Wearing our ODI hats, we launched the new monthly Data Meetups in April – and managed to squeeze in nine of them this year. These are really well attended, and saw over 300 people in total coming out on a Tuesday night to hear speakers from across the country on a diverse range of data topics. These ranged from Creating a Data Culture in your business, to public Open Data; from the data of Scottish Football to the use of blockchain in Oil and Gas; and from the use of IoT in Agriculture to extracting data from photos published on Flickr in order to assist conservation.

Open Data

We’ve also been lobbying the Scottish Government and the city council on Open Data, as Ian has been writing on our sister site. That is starting to bear fruit. Aberdeen City Council have soft-launched a new open data platform, and are recruiting a manager for their open data work. While this is good, it is not as impressive as Dundee and Perth‘s new platforms, yet. The Scottish Cities Alliance are recruiting a new programme manager, and Ian has been invited to be part of a round table discussion on the way forward for Open Data hosted by the Scottish Government next February. It sounds like things will start to move in the right direction in 2019!

Research

Ian and Andrew have worked with ODI HQ to run two local workshops, contributing to two national pieces of research: the first on the effects of Peer to Peer markets on accommodation, and a second on what barriers there are to the better use of Ordnance Survey data and services.

Here’s to an equally successful 2019! Have a great festive break folks!

Ian, Andrew, Steve, Bruce

 

CTC14 Archaeology Weekend – write up

We held our CTC14 Archaeology weekend, which was sponsored by Aberdeenshire Council Archaeology Service, on the weekend of 15 and 16 September 2018.
All code, and some data and documentation which were created over the weekend has been published on Github repos. 

Background

Throughout 2006 an archaeological dig of the East Kirk of the St Nicholas Church was conducted by a team led by the archaeology service of Aberdeen City Council. You can read more of the history here. A large number of skeletal remains and other artefacts were recovered. Written records were created in the form of plans, and log books, and some of these were drawn, then scanned, and a MS Access 2 Database was also created.

Since the end of the dig, some post-excavation analysis of skeletal remains, and other artefacts, has been conducted, but this is far from complete due to a lack of funds.

Saturday – getting started

Following an introduction from Ali Cameron, the dig director, challenges were identified,  ideas for tackling those identified, and teams formed around those.

The teams, and their projects, created a pipeline; one feeding the other.

Below we introduce the teams. Each of these will shortly be linked to individual blog posts for each team.

The Teams

Team Scoliosis

They had two aims:

  • to re-label photos and bringing them together for the ‘skelelocator’ team,
  • to lay out skeletons and explore options for 3D scanning using mobile apps and cameras with photos offloaded to laptops for processing.

Team Skelocator

Working from CSV files (derived from an Access 2 MDB file), JPEG diagrams, Corel Photopaint files and even using original hand-draw plans and log books, this team aimed to create a complete set of data of all excavated remains, allowing these to be plotted in 3D space using X, Y and Z coordinates.

Team Skeleton Bridge

This team set out to create a  schema and mesh diagram which would use the data from team skelocator in a format which the unity burials could use.

Team Unity Burials

The members of this team wanted to create a 3D model of the church interior in Unity, and to place skeletal remains accurately in the 3D space, moving from block models to accurate ones. Their initial focus was on setting up the deployment of their basics to GitHub and to speak to the other teams about what the formats of data they could work with in order to move this along faster

Team PR and Marketing

This team was looking at stories that would help drive any fundraising later, and exploring what data might be possible for visualisations.

Saturday 5pm update

The second round of updates at 5pm on Saturday saw each team make progress.

Team Scoliosis

They were using Qlone for scanning from mobile phones and found it takes only a few minutes per bone – this is also the app recommend for this by Historic Scotland. However, they are blocked by the size of the paper grid that is needed under the object being scanned. An A3 sheet is not quite big enough for larger bones. Another group found it took 20 minutes with laptop to produce low res version from camera photos, and all tried pushing completed models to Sketchfab site.

Team Skelocator

They explored raw files to see what could be extracted with different tools looking at exif data and whether this could be supplemented with data from the dig books if necessary.

Team Skeleton Bridge

It appeared that there was no need for this team as Skelocator could output neutral format files, with agreed content, directly to team Unity Burials. So the team disbanded and members were absorbed into other teams.

Team Unity Burials

They were experimenting with how they can manipulate data from one app to another to provide reference points for when they do have the skeletons to place in the model, and how they might show the metadata of each skeleton too.

Team PR and Marketing

This team were exploring how to use the Microsoft ML libraries to build a chatbot that would use FAQ information about the dig to answer questions.

Sunday morning

The Sunday had teams coming together from 9:30-10:30 and most people returned which was good. We saw an update around noon.

Team Scoliosis

The team used Qlone to scan more bones with mobiles. They discovered the resolution was high enough, even on smaller ones, to be able to notice things which hadn’t previously been noticed such as what appear to be sword cuts to a rib bone (white marks) on a person who was known to have been stabbed in the head.

This highlights the importance of scanning the bodies while they are available before re-internment at a later date. The larger scans with Qlone were found to be too big for detail as you had to stand further away and thus lost resolution.

Team Skelocator

They are now using the Python library Sloth to bring the images to life by extracting text from them and putting this into a JSON file. They also found this enabled a way to position each of the skeletons by marking their locations on the image and then creating a grid for reference from a known fixed location, which could also be used by the unity burial team.

Team Unity Burials

One member worked out a convoluted, but workable, process to get images into Unity from other file formats, while another team member worked on a UI for the public to navigate the model.

Team PR and Marketing (aka skeleton)

They started a chatbot, but found it needs to have its data in a better format, and are starting to work with with cleaning up a list of skeletons for using with dc.js visualisations, as well as a webpage for holding the data from the other teams.

Final presentations on Sunday afternoon – 4pm

Team Scoliosis

Everything is being brought together in Sketchfab so that all models can be found.

https://sketchfab.com/models/6edc98b057c740b5a66d34276ee261da/embed St Nicholas Kirk Skeleton SK820 Skull by Moira Blackmore on Sketchfab

They are exploring how to combine smaller models to create bigger ones by exporting them into another app. They discovered the limits of scaling the grid in Qlone to get the best resolution with devices, and how to use photos and a laptop to get a scan of whole skeleton using the right background cloth.

Team Skelocator

They further pulled data from processed photos with x,y,z locations from a superimposed grid that could be automated with human double-checking to make up for the lack of GPS being available for their tools in 2006, as it is now. This can then be handed to unity burials.

Team Unity Burials

One member found more ways to bring in scanned data from team scoliosis, while the other improved upon the UI for the VR version and demoed the basic model to people with the VR headset.

Team PR and Marketing (aka skeleton)

They updated their google doc spreadsheet and pulled it into the pages at GitHub.io so that it could be queried for skeleton info and finished adding the basic visualisations of this data with dc.js while the skelebot was improved with some personality, but wasn’t as useful as it was hoped it would be.

Take-aways from the weekend

  1. We saw how well cross-functional teams worked as there was usually someone around during the event who could help with something, and that within each team people were able to bring diverse experience to help with issues. This was most apparent during the ‘round ups’ of team effort when people heard what others were doing or trying to do.
  2. We learned that skeletons hundreds of years old, and paper records of 12 years ago ensured durability of information, while a two year old tech gadget from Google was found to be useless after it updated itself as google had stopped the project, so its built in two-camera scanner couldn’t be used. Similarly file formats which were common at the time of the dig, just 12 years ago, were challenging to access.
  3. We also found that these ‘strongly themed’ events work well for participation. We had 28 attendees on Saturday and 25 on Sunday.

Hack and Co-design Events

Harnessing the collective intelligence of crowds can take many forms, and a common version in the tech community is the gathering where people come together to put together potential solutions. This can take many forms, from ‘jams’ in the design community, to the co-design and hack events that we’ll talk about here.

20171125_121721

There is more to this than bringing the right people together. There are the goals of the event. What should the outcome be for the event? How should the event be facilitated? What’s the schedule for the event? The organisers also need to consider where and when the event will be held, as well as how the event should be funded, and if food and other refreshments are to be provided. Then there’s the recruiting participants, and then deciding at various times whether you have/don’t have enough people signed up and what you might do about that to recruit more attendees. Then even when the event has started the issues don’t stop as you need to manage contingencies on the go if people don’t turn up, or the heating disappears for your February event in a cold, snowy climate. Organising events is a non-stop rollercoaster of thrills and spills.

Hackathons, co-design events, what’s the difference?

If this is a collection of whoever comes, then this is a hackathon, or hack event. This generally leads to similar participants on the teams with little diversity. Sometimes these are run for computing students by other computing students with prizes provided by industry as a way to engage students. Sometimes it’s a group exploring what’s possible with the given tech.

If there is consideration given to ensuring a diverse balance between stakeholders (people from organisations involved, and the people who will use the system), who can help define and clarify the challenges, and the community of designers, developers, and tech folks, who are wanting to help prototype solutions, then this is a co-design event. At CodeTheCity we strive to ensure we organise and facilitate co-design events, as we’ll explain below, because these provide a better outcome for everyone.

At a hackathon ideas are proposed and solutions thrown together by the developers based on their assumptions, and interests. This often ignores the human element of the challenge, and the resulting hack remains unvalidated by the community, which is supposed to use this potential solution. Often this looks like ‘a database to gather all local events so you always know what’s going on’. The team then spend lots of time detailing the structure of the database, and the library to use for the interface. This ignores the main social challenges with this potential solution: How will people know it’s there? Who will put new things in, and update changes? Who will weed out incorrect, or duplicate information? More importantly perhaps, this ignores whether this is the challenge facing people trying to work out what to do this coming weekend. It might be a solution looking for a problem. It might be a solution to the wrong challenge as it hasn’t been validated by the people who are proposing the challenge.

Hack events can be fun and exciting, but doesn’t usually lead to solutions that live beyond the event. We have done these for fun as we sought to balance our serious events with more light-hearted ones at Christmas time where people brought their own challenges for the weekend, or we explored a new type of technology such as chatbots. The results have all sat there unused, unless it was a hack started by someone before the event.

A CTC Workshop
CTC Workshop

We use the phrase co-design to express the idea that it is a collective, or communal process to design a potential solution to a specific challenge offered by the stakeholders. We want to include most of the relevant stakeholders in seeking to develop a solution on how we might address their challenge. We want to validate our assumptions about the challenge posed by the stakeholders as we co-create a solution with our participants. We want to find out how people currently decide what to do at the weekend as a starting point, and build on these ideas to determine what a better solution would include, and then explore a number of different ways in which we might achieve this goal.

Then, and only then, do we start to develop a prototype. We seek to spend more time exploring the challenge space before diving into a potential solution. We give in to our curiosity to explore the challenge more, and to generate a number of possible options, which can be evaluated to see which one offers the ‘best’ potential given what we know at the time. I’ve described this process elsewhere if you want to know more about how to ‘diverge’ and ‘converge’ for creativity.

The goal in all of this is to build potentially workable solutions to challenges so that the stakeholders can see which of the potential options they had is the most viable. A co-design event helps to clarify the unknowns around potential solutions so that knowledge is advanced. Now the stakeholder knows that one idea needs further thought, another could work, and that what looked like it might be a great idea, was really an illusion. All of this for a weekend of food, drink and fun. Not a bad exchange really.

The participants come for other reasons. The have a chance to work on real challenges with new people. They have an opportunity to stretch their abilities and gain new skills. The food and drink are nice, but they’re not the real reason people come. They come to see friends, make new ones, and to help make the world a better place thanks to their hard work over a weekend.

As organisers we enjoy helping facilitate the finding of solutions to the challenges brought to our attendees by our stakeholders. We enjoy seeing the diverse group of people come together from developers, designers and our stakeholders. It’s especially pleasing when we see some of these apps and ideas making their way into the world. We also enjoy seeing repeat attendees developing their skills and friendships over time too.

If you think a co-design event would benefit your organisation, or if you are struggling with intractable problems and think that a fresh approach could help, then get in touch via an email to bruce@codethecity.org

When data glitches ruin your day

The Scotrail service between Aberdeen and Inverness is currently disrupted by some improvement works to the line between Aberdeen and Dyce. Bus replacement services are in operation between Aberdeen and Dyce, with normal service between Dyce and Inverness. All because the line has been dug up in preparation for new dual tracks being laid.

But if you do a search on the Scotrail website for the next train north from Aberdeen you get this message:

Train_Tickets___Times___Timetables___Fares_in_Scotland___ScotRail

“No services between these stations have been found”. One would assume that Scotrail isn’t running services from Aberdeen to Insch. The same thing happens for searches to all stations north of Aberdeen. Continue reading When data glitches ruin your day

Cantcha @ codethecity 13

During Codethecity 13, a project arose around the clarity of labelling of alcohol, initiated following a presentation from @wayne_gault about alcohol labelling and marketing.

Quickly the focus of discussion moved to craft beer, and the rising importance of graphic design in the branding and positioning of the various craft breweries, often using cartoon and other colourful illustrative styles. In a competitive market with many new entrants it’s no surprise that some corners are being cut. Sometimes through a lack of awareness of the issue, sometimes not so much.

Sending one particularly colourful can to the Google Vision API was telling. Google identified it as a “Yellow, Aluminium Can, of Soft Drink”. You can try the vision API yourself here.

Vision_API_Google_Cloud

The team working on this issue came up with the idea of creating a new version of reCaptcha. Rather than clicking on all the store fronts, or vehicles, you would click on all the beers, or all the energy drinks.

This would both raise awareness of the issue, and provide a data set to demonstrate the level of confusion around specific labels. If the ‘spongebob’ beer in the mockup below is regularly identified as an energy drink, perhaps the labelling falls short of the standard.

Screen Shot 2018-06-09 at 19.47.22

How this could be useful

The Portman Group publishes the Code of Practice for the naming, packaging and promotion of alcoholic drinks. An ultimate goal of the CANtcha team was to use data generated from the app to add weight to submissions to the Portman Group where packaging falls short of the Code.

Some early (and far from rigorous given the timescales) testing showed that enough of the alcohol was being misidentified that an issue likely exists.

Code for the mock ups, some initial data sets, and some initial analysis are available on Github. A more technical write up is also available.

Even though this was a very brief, short lived prototype – I’ll never look at a beer in the same way again

I subscribe to a craft beer delivery service. It sends me a case every now and then with enough interesting beer from around the world to keep my ‘bottle of hipster a week’ habit going.

A case just arrived.

Now that I’m aware of the Portman rules, I couldn’t help wonder what CANtcha would make of a couple of them:

IMG_20180607_193824

IMG_20180607_193738

I suspect both of these might fall short on one or more counts. I also love both of them. The Google API mentioned above identifies one as beer, and the other as an energy drink. It’s likely obvious which is which.

Neither is identified as a can of coffee, surprisingly.

Volunteer with us

At Code The City we are always looking for opportunities to work with volunteers.

We recognise that this not only helps us to delivery on our charity’s aims, but also helps the volunteers themselves.

We’ve just published an overview of the opportunities, with a new volunteer form, and mention of the upcoming ‘I volunteered at Codethecity’ t-shirts. Guaranteed to be the must have fashion item of 2018.

So if you’d like to help us at upcoming hack weekends, please check out that page and fill out the form.