Die Demokratie Suchmaschine

We will be at the Scraping Day(s) run by the very good people of ScraperWiki as part of the pre-programme of the Open Knowledge Conference in Berlin. We already use ScraperWiki for scraping a number of data sources.

Here is what we do:

1. We write a a bit of code on ScraperWiki that scrapes some information about projects relating to political participation. For each project, we are interested in the following pieces of information:
   • a telling title
   • a description of what it is about – but not too much, as meine-demokratie.de is only meant to refer to the original sites for detailed information, therefore we also need
   • a URL to the original content
   • the name of the data source
   • a place - this can be anything from an address, a zip code, a city or even a election constituency
   • a category - classifying what type of project we are looking at, e.g. a demonstration or else
   • optionally, you can add a start and and end date as well as tags
   • for a real world example, have a look at our ScraperWiki code with which we get information about surgeries of some local representatives from the official calendar of berlin.de
2. When the data is collected, we output it in an XML format that we can feed to our platform. The basic format is defined here. In ScraperWiki, we achieve this by using a customised view – here is again the example of berlin.de.
3. Once it is collected, we import it to our platform and its visualised there immediately.
4. In the process, we also add lots of useful additional information to the data, including post codes, council, election district and so. This can then be exported again in an extended RSS format – here again shown with the data for berlin.de that was originally collected with ScraperWiki.

Our plan for the Scraping Day

We want to cover many more practical opportunities for citizens to get involved in politics and have their say. The data available is dispersed and in multiple formats, but with the help of other people at the Scraping Day we hope to get some more of this information in a standardised form to make it searchable and accessible via meine-demokratie.de.

Here is a list of stuff that we would like to see on the site as it will be valuable to others:

1. election dates: from Wahlrecht.de (nice to have too: list of past federal state election dates )
2. list of citizen referenda on state level: from Mehr Demokratie e.V. (current and past)
3. planning permissions in Berlin:
   1. from state: Bebauungspläne
   2. from councils: 12 different sites, e.g.
      1. currently running consultations in Treptow-Köpenick
      2. past and present consultations in Charlottenburg-Wilmersdorf
      3. current consultations for Mitte
      4. past consultations for Tempelhof-Schöneberg and current ones
4. list of participation projects from buerger-beteiligung.org
5. list of citizen foundations in Germany
6. your ideas here …

Looking forward to the scraping day!