Goodness, is that the time?!
Our teams have been working away for a few hours. So, it’s time for an update. If you have missed the intro to this weekend’s event, read first here.
We kicked off the ‘Code The City 9 – Health Signposting’ weekend this morning bright-eyed and bushy-tailed. There are just under 20 attendees from mixed backgrounds.
We have volunteered to help solve issues around health care data. One problem is that health care data are currently maintained in (at least) three unconnected systems run by different organisations. These are ALISS, GCD (Grampian CareData) and MILO. The ultimate goal of this project is to create an open data source that provides accessible up-to-date information to the public and professionals.
After two days of intense activity and a whole heap of learning for all of us, Code The City #8, our Chatbots and AI weekend came to an end at tea time on Sunday.
It couldn’t have happened without the generous sponsorship of our two sponsors: The Health Alliance, and Fifth Ring, for which we are very grateful.
The weekend rounded off with presentations of each project, four of which we’ve captured on video (see below).
Each of the projects has its own Github repo. Links are included at the end of each project description. And, two days later, the projects are still being worked on!
Team ALISS worked on providing a chatbot interface onto healthcare and social data provided via the ALISS system.
You can find Project ALISS’s code here on Github.
You can also watch this video of Douglas Maxwell from the Alliance being interviewed about the weekend (although at the time of writing the video is offline due to an AWS problem).
This team aimed to make the quality of consultations better through using intelligent chatbot interfaces to guide users through the process – and to provide challenge by prompting citizens to comment on previous consultees’ input.
You can find the code for City-Consult at this Github repo.
The concept for NoBot came from an initial idea which was of a bot which would make scheduling meetings easier. That spawned the idea – what if the Bot’s purpose was to make you have fewer meetings by challenging you at every turn, and in the process the bot’s personality as a sarcastic gatekeeper was born.
The code for Nobot lives here on Github.
Sadly there is no video of the wind-up talk for Seymour. In short the purpose of Seymour is to help you keep your houseplants alive. (More details to come).
You can find the code for Seymour at this repo on Github.
We started this project with the aim to help citizens find out what was happening in the myriad of local events which we each often seem to miss. Many local authorities have a What’s On calendar, sometimes with an RSS feed. None we found had an API unfortunately.
We identified that by pulling multiple RSS feeds into a single database then putting a bot in front of it, and either through scripting or applying some AI, it should be possible to put potential audiences in touch with what is happening.
Further, by enhancing the collected data – enriching it either manually or by applying machine logic, we could make it more easily navigable and intelligible.
Expect a full write-up of the challenges of this project, and what progress was made, on Ian’s blog,
There is no video, but you an find the project code here on Github.
This project set out to solve the problem of checking if a shop or business was still open for the day through a Facebook bot interface – as you with wander around, wondering about the question, as it were.
You can find their code here.
And finally we were joined by Rory on day two who set out to assist team Stuff-Happens through developing some of the AI around terminologies or categories. That became the:
This is now on Github – not a bot but a set of python functions that scores a given text against a set of categories.
We had loads of positive feedback from those who attended the weekend (both old hands and newbies) and from those who watched from afar, following progress on Twitter.
We’ve published the dates for CTC9 and subsequent workshops on our front page. We hope you can join us for more creative fun.
Ian, Andrew, Steve and Bruce
This post was originally published on 10ml.com by Ian Watt
The art of scraping websites is one beset by difficulties, as I was reminded this week when re-testing a scraper that I built recently.
As part of my participation in 100 Days of Code I’ve been working on a few projects.
The first one that I tackled was a scraper to gather data from the PDF performance reports which are published on a four-weekly cycle Scotrail’s website. On the face of it this is a straightforward things to do.
So, I built the bones of the scraper in a few hours over the first couple of days of the year. I tested it on the then current PDF which was for period nine of 2016-17. That worked, first creating the clean CSV, then later adding the DB-write routines.
I then remembered that I had downloaded the previous period’s PDF. So I modified the code (to omit the downloading routine) and ran it to test the scraping routine on it – and it blew up my code. The format of the table structure in the PDF had changed with an extra blank link to the right of the first list of station names.
After creating a new version and publishing that, I sat back and waited for the publication of period 10 data. That was published in the middle of this week.
I re-ran the scraper to add that new PDF to my database – and guess what? It blew up the scraper again. What had happened? Scotrail had changed the structure of the filename of the PDF – from using dashes (as in ‘performance-display-p1617-09.pdf’) to underscores (‘performance_display_p1617_10.pdf’)
That change meant that my routine for sicking out the year and period, which is used to identify database records, broke. So I had to rewrite it. Not a major hassle – but it means that each new publication has necessitated a tweaking of the code. Hopefully in time the code will be flexible enough to accommodate minor deviations from what is expected without manual changes. We’ll see.
Of course, none of this should be necessary.
In a perfect world Scotrail would publish well structured, machine-readable open data for performance. I did email them on 26th November 2016, long before I started the scraper, both asking for past periods’ data and asking if they wanted assistance in creating Open Data. I got a customer service reply on 7th December saying that a manager would be in touch. To date (15 Jan 2017) I’ve had no further response.
Abelio operates the Scotrail franchise under contract to the Scottish Government.
Should the terms of such contracts not put an obligation on the companies not only to put the monthly data into the public domain, but also that it be made available as good open data – and follow the Scottish Government’s on strategy for Open Data ? Extending the government’s open data obligation to those performing contracts for governments would be a welcome step forward for Scotland.
Code the City #8, which will take place in on Sat 25th to Sunday 26th February 2017, will be an exploration of the world of chatbots and AI (or Artificial Intelligence), identifying problems to tackle and quickly prototyping solutions.
>>> Book a ticket on our Eventbrite page
A chatbot is a piece of software that interacts with a customer or user to directly answer their questions. It uses existing data or information coupled with artificial intelligence to respond in a human-like way, guiding the user to a solution.
There are many examples of live chat bots in this exciting, emerging field. A chatboat could give you travel directions, tell you when its next going to rain in your area, or help you contest parking tickets. It could book you a flight and hotel, or act as a free lawyer to help the homeless get housing . The HBO series Westworld has even launched a bot to help you interact with the (fictional) holiday park!
If you are new to this field and want to get started we suggest you read the Complete Beginners Guide to Chatbots (and some of the links at the end of this article).
We’ll apply our usual Code The City methodology:
You will create mixed teams to workshop chatbot solutions to real world issues. Maybe these will building on the outputs of previous work we’ve done at CodeTheCity. Through rapid prototyping you will create new applications and have some fun in the process.
We’ll show you new techniques for service design, idea generation, prototyping, and rapid iterative application development – and you will show other participants some tricks and approaches, too. We’ll share knowledge and learning.
You might even get a Tshirt, and we can guarantee the best catering of any weekend workshop in the city!
To book a free ticket visit our Eventbrite page But be quick, tickets will go swiftly!
All attendees will get a year’s free membership of the Open Data Institute.
If you have any questions please get in touch.
If you are interested in sponsoring this event please, or providing other support such as access to online tools or services, please get in touch.
>>> Book a ticket on our Eventbrite page
Ian writes about using open data to look at GP appointment availability on the Aberdeen ODI Node blog.
“We should build one for here!”
So starts another Codethecity conversation on discovering a neat data driven tool. This time it’s the excellent New York subway toy created by Jason Wright.
The tool allows you to redesign transit provision in the city by building new subway routes. By adding new stations. By removing or moving existing lines.
It’s addictive and fascinating.
As is so often the case, we then start riffing on what it could also do. It could time travel using that tram data we have from the early 1900s. It could give alternate route options if we hook up to that academic project we spoke with earlier in the year. It could carbon count. It could give safety information for cyclists. We could data collect with a new app to feed it improved validation data…
Before we have the cake we’re discuss how pretty the icing will look.
In reality what we should be looking at is the bottom layer. The underpinnings. The data.
Where do people live? Where do they work? Where do they school run? Where is the football stadium and where do the fans live? Where are the shops and where is the money?
We’re going to start with the commute. Where do people start, spend, and end their day? How do they move around? And when? No agenda. No grand insights planned. Just a good solid data gathering and modelling project.
We’re calling it journeygrid.
If you have any data, or methodologies for gathering and storing such data we’d love to speak to you.