Cork City – Street Types

I was reading Alasdair Rae’s street types blog post yesterday and I thought about doing something similar for Ireland. I didn’t have much time today so I said I’d put something quick together for Cork City. Alasdair was lucky that he could use the amazing open data from the UK’s Ordnance Survey. For Ireland, we don’t have anything similar from our national mapping agency but fear not because it’s OpenStreetMap to the rescue. In my humble opinion individuals often spend far too much time trying to get a usable output from overpass-turbo when there’s an easier way. What I did was download the .osm.pbf file for Ireland from Geofabrik. Windows Subsystem for Linux has been a lifesaver since it came along, most of the heavy lifting I can now do through it. The steps I followed are as follows:

1. Download the .osm.pbf file using wget.

Wget Download

Wget Download

2. Install the osmium-tool

Osmium-Tool

Osmium-Tool

 

 

 

 

3.Filter out just the ‘highways’ from the dataset and save it to a new .osm.pbf file

Filter Highways

Filter Highways

 

 

 

 

4. Convert the .osm.pbf file to a new shapefile ( I don’t normally use shapefile but converting to a geopackage was causing some issues in this instance).

Save as Shapefile

Save as Shapefile

 

 

 

 

 

5. Reproject the shapefile to EPSG:2157 (Irish Transverse Mercator).

Reproject to ITM

Reproject to ITM

 

 

 

 

 

 

I could have made this more efficient again by combining some of these actions (such as save as a new shapefile and reprojecting), I did it this way to make the process easier to explain.

Next I brought the dataset into QGIS and extracted an area just for Cork City. The main issue I then has was how to ascertain what road types occur most frequently. I exported the Cork City extract as a CSV and imported it into Excel. I then used a generic formula from the folks at exceljet to extract the last work from the ‘name’ column. I then counted the frequency of the extracted work in a pivot table to get an idea of the types I’d like to use.

Frequency of Road Types

Frequency of Road Types

 

 

 

 

 

 

The last thing I did was to create a new field using an expression to find the various road types in the ‘name’ column from the OSM data. The expression I used is the below:

CASE
	WHEN "name" LIKE '%Road%' then 1
	WHEN "name" LIKE '%Park%' then 2
	WHEN "name" LIKE '%Street%' then 3
	WHEN "name" LIKE '%Avenue%' then 4
	WHEN "name" LIKE '%Court%' then 5
	WHEN "name" LIKE '%Drive%' then 6
	WHEN "name" LIKE '%Hill%' then 7
	WHEN "name" LIKE '%Lawn%' then 8
	WHEN "name" LIKE '%Estate%' then 9
	WHEN "name" LIKE '%Place%' then 10
	WHEN "name" LIKE '%Lane%' then 11
	ELSE 0
END

I then symbolised the data and exported it. The final product is below. Obviously this was a pretty quick and dirty way to do it and if I create an atlas for every major town in Ireland I will use PostGIS which will allow me to count the frequencies and create the new field with ease.

Cork City - Road Types

Cork City – Road Types

Unpopulated Areas

I was hiking at the weekend and it got me thinking about the unpopulated areas of Ireland. I’ve seen maps made for the 2011 census showing the square kilometres that have no usual resident population but I hadn’t seen one for the 2016 census so I put together the below. I purposefully omitted Northern Ireland because the data is nine years old. If anybody would like the replicate the below just leave a comment and I can do a YouTube tutorial or post on here on how I put it together.

 

The Unpopulated Areas of Ireland

The Unpopulated Areas of Ireland

Development in Ireland

A topic that I think about frequently is the physical changes Ireland has gone through over the last 10 or 20 years. I’ve covered it before in this blog but I’m going to look at it again today. I think you could ask anybody around the country where the development in their town has been during the Celtic Tiger and this current boom period (well, up until the Covid-19 outbreak anyway) and they would easily be able to tell tell you.

It’s true that we have national indicators from the Central Statistics Office identifying where the most planning permissions have been granted as well as data from the Department of Housing, Planning and Local Government on new ESB connections to show where new homes have been constructed. I was looking for something that would give an idea of the physical development that has occurred in the last 20 years.

Enter the CORINE Land Cover Inventory. It started in 1985 as a way for the European Environment Agency (based in Copenhagen) to monitor the main land uses in the Union. As a little factoid, it stands for Coordination of Information on the Environment. It contains 44 land use classes and the cell size is 100 metres. It already contains layers that show the change between different years but it didn’t contain any change layer from 2000 to 2018. I wasn’t interested in the 44 classes, in fact I was only interested in the categories shown below. I decided to omit construction sites, dump sites and a few other inconsequential categories. These are the main categories that will show development in Ireland between those years.

111 – Continuous urban fabric
112 – Discontinuous urban fabric
121 – Industrial or commercial units
122 – Road and rail networks and associated land
123 – Port areas
124 – Airports

I used the the wonderful r.reclass GRASS tool in QGIS in order to reclassify the rasters for both the year 2000 and the year 2018 to combine these categories. I then used raster algebra in order to subtract the year 2000 from the year 2018 raster in order to show the actual change. Below is a GIF showing the process for an area of North Kerry and Cork.

QGIS Process Example

QGIS Process Example

I create the below map in A1 so you can zoom in to get a good sense of what 18 years of development looks like in a particular county.

Ireland-Development Between 2000 & 2018

Ireland-Development Between 2000 & 2018

Here is a link to the PDF version if you would rather.

*Update*

I redid the map showing the development up to the year 2000 as a separate colour. I’ve also added a legend.

Development in Ireland

Development in Ireland

Here is a link to the PDF of this version.

Finally, after a few unsuccessful attempts at converting the final raster to vector I used the gdal_polygonize tool which did it seamlessly without any loss of the smaller cells. Below is the final layer as a web map that you can make full screen.

There are a few interesting area that were developed in those 18 years such as Aughinish Alumina in Co. Limerick:

Aughinish Alumina

Aughinish Alumina

There are also a few windfarms such as this one in Tyrone:

Windfarm in Tyrone

Windfarm in Tyrone

Peatlands of Ireland

I was thinking the other week about the amount of bogs around Europe after the collapse of a trial against turf cutters in Galway. I put the below map and tweet together in a few minutes that shows the peatlands of Europe. The outlines of Iceland, Ireland, Northern England, Scandinavia and Scotland are visible from their peat cover alone. The data comes from the European Commission’s CORINE land cover dataset.

After completing the above I thought I’d put together a map showing the peat cover in Ireland. Besides from the CORINE land cover dataset I used a DEM from the European Space Agency to create the hillshade and the administrative data comes from Ordnance Survey Ireland and OpenDataNI. The dataset doesn’t differentiate between raised and blanket bogs but for my purposes that’s okay.

Below is the finished map. I tried something new and used abbreviated county name labels à la various maps for the USA where the state names are abbreviated. I think it works well.

Peatlands of Ireland

 

I was thinking since I posted this that there’s a bit of work involved to figure out the order of peat per county so I created a graph. For any GIS folks out there, I quickly unioned the county layer and the peat layer and then calculated the area in square kilometres for the peat and exported this as a CSV. I then used Matplotlib to create the below graph. If you’d like to know more about simple workflows like these just reach out to me on Twitter (@pearoid).

Peat Area per County in km²

Airbnbs in Ireland

I was reading this Guardian article the other day where they produced maps showing the number of Airbnb listings per 100 dwellings. I thought it was really interesting and I hadn’t seen Airbnb data mapped like that before. I had a few hours to spare yesterday so I set about replicating their method for Ireland. I used the 2016 census electoral divisions (to get the household numbers) and data for Ireland from Inside Airbnb.  I think at best this data is questionable because from the reading I’ve undertaken it seems to still list properties that were briefly on Airbnb a number of years ago and have long since been removed however this is the only data available so I went with it.

Below is the map, it was made with a combination of Bash, GDAL, QGIS, LibreOffice Calc and Illustrator.

Airbnbs per 100 Dwellings in Ireland

Airbnbs per 100 Dwellings in Ireland

Irish Census 2016 & Privacy

I’ve been looking at the 2016 census results with the last few years and there is a great deal of suppression of values for relevant Small Areas. The CSO suppress results or aggregate them depending on the number of people living in a Small Area. If the population is too small and could lead to individuals being identified, the data is suppressed. They are legally required to undertake this exercise under s33 of the Statistics Act, 1993.

I’ve been looking at a selection of variables and after reading this piece on the traveller accommodation crisis by RTÉ I decided to map the percentage travellers per Small Area. I have all this data in a PostGIS database but I’ll quickly run through how to do it without having to use PostGIS. I downloaded the Small Areas shapefile (generalised to 50m) and the CSV of all of the Small Area values from the CSO here. Instead of having to use a spreadsheet or QGIS to manually delete the 802 fields I didn’t need I used the pandas library, the python code below that took 0.3 seconds to run. It opens the relevant CSV and only selects the columns that I need and then strips the first 7 characters from the ‘GEOGID’ string as these are not needed for the join I’ll do in QGIS later.

import pandas as pd, time

start = time.time()
df = pd.read_csv('SAPS2016_SA2017.csv', usecols=['GUID', 'GEOGID', 'GEOGDESC', 'T1_1AGETT','T2_2WIT'])

df.GEOGID.apply(str)

df['GEOGID'] = df['GEOGID'].str[7:]

df.to_csv('SAPS2016_SA2017_New_GEOGID.csv')
end = time.time()
print(end - start)

I then opened the shapefile in QGIS, imported the CSV and joined them. This was  subsequently exported to a GeoPackage and I used GDAL’s ogr2ogr library to convert it to a GeoJSON in order to upload it to Carto.

ogr2ogr command to convert to GeoJSON
ogr2ogr command to convert to GeoJSON

Below is the resultant map with some formatting of headings undertaken to make it more legible. You can make it full screen using the button on the left. What struck me about this was how with a small amount of work it was very easy to visualise accurately the resident locations of one of the most vulnerable groups of society. Obviously this information is useful to local governments, state government agencies, NGOs and so forth but I question whether this data should been available to the general public regardless of it being aggregated to the Small Area geography.

 

Ireland’s Social Housing

Housing and all its intricacies have come to dominate the media discourse at home over the last few years. We’ve truly come out the other side of the recession and now the conversation is around the shortage of housing and where that has lead us. I’ve been thinking about this recently and in particular social housing. I think that most people assume that we built the majority of our social housing in the 1950s and 1960s. Collectively I think we assume we know when social housing was built but not where. This is where the census data can come in. Part 2 of question H3 in census 2016 asks ‘If renting, who is your landlord?’.

I have all of the census 2016 returns for each geographical unit in a PostGIS database so it was a simple exercise to add the households that rent from a local authority or voluntary/co-operative housing body and divide by the total number of houses. One inherent weakness to this method is that it doesn’t capture the social housing tenants that rent from a private landlord.

Ireland Census 2016-Question H3

I added a new column in PostGIS for the percentage social housing and I then symbolised this in QGIS. I used QGIS’s powerful Atlas generation tool. You’ll have to excuse the basemap, I’m aware that it’s a bit difficult to discern but in the interest of producing this entirely with opensource software I used OpenStreetMap as the basemap.

The next step will be to take the top-ten counties and use the Global Human Settlement Layer as a base to give an approximate indication of what epoch they were built in.