I was looking at a map highlighted on Reddit the other day from the website mapsbynik. It showed the unpopulated areas of the United States. I was going to do a similar exercise for Ireland but when I examined the census data it quickly emerged that no ‘Small Area’ (the unit with the finest spatial resolution) was constructed to have a population of zero, even the Small Areas that formed part of large national parks were considerably larger than the average small area, one assumes this was a conscious decision taken by the Central Statistics Office.
Instead I decided to focus my attention on Australia, from perusing their website, their equivalent of the Irish Small Areas was the ‘Mesh Block’ which were ‘the smallest geographic region in the Australian Statistical Geography Standard (ASGS), and the smallest geographical unit for which Census data are available’ (more information here). In 2011 there were 347, 627 mesh blocks for the country. I created a map of the unpopulated mesh blocks but it wasn’t very informative, large sections of the country were showing as being populated when in fact they weren’t.
In an effort to improve my map I continued searching their website until I found the 2011 Population Grid. The methodology of how their created their population grid is here. I then downloaded 9 shapefiles and merged them together in QGIS. I then used a definition query to select all the cells with a population of zero and created a map from this. It is a vast improvement from my initial effort. I spent some time styling my map in QGIS’s print composer. The below is the finished product. Very stark contrast between the big cities and the outback.
In my last post I downloaded all the McDonald’s in Ireland using overpass-turbo. The problem I had when I exported these was that because GeoJSON is such a flexible format, I was getting both nodes and ways. These would have had to be imported into ArcMap separately and the centroids of the ways found and merged with the points, this would have taken some time. Instead I got a great suggestion “Mmd Osm” on Youtube, that was to use ‘out centre;’ instead of:
out skel qt;
This worked as promised and the centroid of both the nodes and the ways was exported. I then used OGR2OGR to convert this to a shapefile:
ogr2ogr -nlt POINT -skipfailures McDonalds_Ireland.shp McDonalds.json
I had to rename the GeoJSON file to .json for the purposes of this conversion, the last step I undertook was to transform the shapefile from WGS84 to the Irish Transverse Mercator, EPSG 2157. This was completed using the following code in OGR:
I did a bit of quick work today to figure out what’s the farthest you can be from a McDonald’s in Ireland. I used the usual method that I’ve followed in a few previous posts, I calculated the euclidean distance from the existing McDonald’s, that was the OSM Overpass-turbo GeoJSON that I downloaded and used OGR2OGR to convert it to a shapefile. I had planned on using PostGIS for this and St_Distance to find this out but I went with the tried and tested method. The farthest you can be from a McDonald’s in Ireland is Co. Galway, more specifically in the townland of Ballinaleama. It is 75.3km from the nearest McDonald’s at the Westside Shopping Centre in Galway.
I was driving past a McDonald’s in Ireland last week and it got me thinking about the farthest distance you can be from a McDonald’s in Ireland. This has been attempted by others before me for the UK and the United States etc. but I can’t find anything on Ireland so I will give it a go. This post is just a quick post on how to download the relevant GeoJSON file. I will use overpass-turbo.eu to download the McDonald’s locations.
I then used the wizard to look for McDonald’s in Ireland, created a bounding box for Ireland and ran the query, the below was the result.
McDonald’s Locations in Ireland
This was then exported as a GeoJSON file, in my next post I’ll load this as a table into a PostGIS database and find the farthest point from a McDonald’s in Ireland.
I spent a month over Christmas with my other half and her family in Western Australia, I had my trusty (albeit pushing on in years now) Garmin GPS 60CSX on me and used it to record a track of a day we spent on a chartered fishing excursion. I brought the .gpx into ArcMap and saved it as a shape which I then converted to a geojson file using ogr2ogr.
ogr2ogr -f GeoJSON fishing.geojson Fishing_1.shp
I then decided to a create a simple webmap of the trip using leaflet. After successfully creating the map I realised that the ocean basemap that I used for that part of Western Australia didn’t have enough detail so I redid the whole thing using ESRI leaflet. I loaded the geojson externally from a file and added a popup with the distance travelled.
After my last post on the farthest distance you can be in Ireland from a road I got thinking on what the situation would be like for other countries. From what I’ve found in Europe, the situation appears to be quite similar, in England it’s 7.6km, (it’s a cached version of the site as it seems to be offline at the moment). So, logic would dictate that the larger the country, the more interesting the answer.
With this in mind I instantly thought of places like Alaska, Russia and Australia. For various reasons (least of which my other half is Australian) I picked Australia. The first problem that I had to overcome was where to acquire the data, there were two reliable sources that I found, the first was Geodata Topo 250k Series 3 data published by the Geoscience Australia, I had to be wary of this as it hasn’t been updated since 2006. This is the data set that is used to create the 1:250,000 scale topo maps of Australia, it was missing a lot of road details in urban areas.
The next data source was the ever dependable OpenStreetMap. The best source for this data is Geofabrik, a German company that has various data formats to choose. I’ve used their website numerous times in the past with great success but this time it would be different because I wouldn’t be dealing with a small country in Europe but the geographic data for an entire continent. As I wanted to reuse the methodology I had used last time for finding the most remote point in Ireland I had to add the Australia Road data from OSM to ArcMap. The easiest way to do this was to download the OSM data in osm.pbf format (Protocolbuffer Binary Format, intended as an alternative to XML). The free OSMconvert command line utility was then used to convert it to .osm format using the following code:
Once this was converted (one of the advantages of OSMconvert is how fast it is, it took 58 seconds) the OSM Toolbox was used in ArcMap (added from here) to add the .osm file to a file geodatabase. This geodatabase is 12.1GB in size. The roads layer was added (1,513,287 features); the roads that would form the basis of this selection were then selected from this and exported to a few feature class. The select statement is below:
highway = 'motorway' OR highway = 'motorway_link' OR highway = 'primary' OR highway = 'primary_link' OR highway = 'raceway'
OR highway = 'ramp' OR highway = 'residential' OR highway = 'residential_link' OR highway = 'road' OR
highway = 'secondary' OR highway = 'secondary_link' OR highway = 'tertiary' OR highway = 'tertiary_link' OR highway = 'trunk'
OR highway = 'trunk_link' OR highway = 'turning_circle' OR highway = 'unclassified' OR highway = 'track'
I thought long and hard about whether to include or exclude the tracks, in the end I decided to include them because I ran a version of this without them and realised that it omitted public roads like the Canning Stock Route. This is classified as a track and it is located in the northern part of Western Australia and is 1850km long, to put that in European context, that’s about the distance between Dublin and Rome. I then projected the roads to GDA_1994_Australia_Albers (EPSG 3577). I’m no expert in Australian coordinate systems but this seems to be widely used for the country. I then used the Euclidean distance tool to calculate the Euclidean distance for the exported road network. Everything for this exercise was saved in a file geodatabase, this made running the tools faster. Below is an export of the Euclidean distance raster with the road network overlaid.
Euclidean distance raster with road network overlaid.
For the purposes of this exercise all of the islands (Tasmania included) were removed. The next step was to use the zonal statistics tool to calculate the max value of the raster. The following equation was then used to result in an output with only one cell value (the maximum distance to a road). This was then converted to a point using the ‘Raster to Point’ tool in ArcMap.
The maximum distance that you can be to a road in Australia is 156km.
Farthest Point from a road.
The point is located in the Great Sandy Desert. It is the second largest desert in Australia and encompasses an area of 284,993 square kilometres.
Great Sandy Desert
So there you have it, the farthest point you can be from a public road in Australia is almost the distance between Dublin and Galway. Obviously this was just a quick desktop based exercise and has flaws ranging from data quality to the coordinate system used but I think it is useful enough to be a semi-decent indicator of the distance.
I recently read an article by Mark Jenkins in the Guardian celebrating 100 years since the foundation of the US National Parks Service. In this article he seeks the help of a GIS specialist to find the most remote place in the contiguous United States. It turns out that this point is just over 20 miles from the closest road. I strongly suspect that publicly accessible land must have been one of criteria as the centre of the Nevada Test Site (Wiki article)is certainly farthest than the point they found. I have emailed the GIS specialist and asked him the criteria he used. I’ll update this post if he responds.
OpenStreetMap view of the Nevada Test site.
This got me thinking, if for an area as large as the contiguous USA (8,080,464.3 km2) the distance is so small what would it be for Ireland given that Ireland is 95 times smaller than it (8,080,464.3/84,421=95.7). I did some research to see if any greater minds than my own (read: ALL) have undertaken any research on this. The first (and in my opinion the most reliable) source I found was the United States Geological Survey (USGS). They have published a brief paper (link) where they give the distance to nearest road every 30 metres for the lower 48. They have also published an academic paper (Roadless Space of the Conterminous United States) that can be viewed for free if you register.
The first thing to do was to acquire the administrative area of Ireland (for this I used the OSI’s landmask shapefile). The next step was to download all the rivers and lakes of Ireland (these were downloaded from the EPA’s geoportal). The final step was data acquisition was to download the OpenStreetMap road network from Geofabrik. Once all these were downloaded, I could begin.
The first task was to use remove all features from the roads shapefile that contained attributes not relevant for this exercise (any non-publicly accessible road). The features removed were:
The above features were selected from the roads layer in ArcGIS as shown below.
Select by Attributes
The next step was to invert this selection and export it to a file geodatabase, 103,808 attributes were non suitable for this analysis. Once this selection was exported it was added to the map. A problem faced at this stage was that all of the roads for Northern Ireland were included. There were two maps to approach this problem, the first was to clip them to the extent of the Republic shapefile or the other option would leave them be and they would not be included by default when the zonal statistics was run as the zone would be the Irish shapefile. The second option was chosen for simply time saving reasons, from previous experience, a clip of the roads would take quite some time.
As all the data that was downloaded was in the geographic coordinate system WGS84 it had to be reprojected into TM65 Irish Grid (EPSG 29902). Everything was then exported to a file geodatabase to speed up future calculations.
Reproject in ArcMap
The next step was to calculate the Euclidean distance for the roads layer. This was done using the Euclidean distance tool in ArcMap. The main variable here is what cell size to use, obviously the smaller the cell size the more accurate result but this comes at the expensive of processing time and file size. I chose a 30m cell size as I felt this would strike the right balance. This operation ran very quickly and the resulting raster with the roads overlaid is shown below.
Euclidean Distance Raster
With this raster each cell has a value based on its Euclidean distance from the road polyline. For the purposes of this exercise, the islands were discounted as I was only interested in the main landmass of Ireland. The islands were removed. The next step was to calculate the zonal statistics, the zone in this case was the Ireland polygon and the statistic that was calculated was ‘MAXIMUM’; this would tell us the cell that was the maximum Euclidean distance from a road.
The zonal statistics tool creates a new raster with just the maximum value present. This raster was then used in the following equation to calculate the locations where the distance equals the maximum distance.
Con("EuclideanD" == "Zonal_Max","EuclideanD")
This then generates a raster with only one value, this is then converted to a point using the ‘Raster to Point’ tool in the ArcToolbox. The result is shown in the image below. The farthest point from a road in Ireland is 6,760m from a road, it is in Nephin Beg Mountains in Ballycroy National Park in Co. Mayo, very close to Slieve Carr. To put this in perspective, the farthest you can be from a road in Ireland is the same distance as between O’Connell Bridge and Farmleigh Park to the west of the Phoenix Park, a very short distance indeed!
There are two drawbacks to my methodology, the data used from OSM is only as accurate as its contributors and I didn’t clip out the lakes and rivers of Ireland. I purposefully didn’t clip out the lakes and rivers because the end result would not have altered (as long as the point wasn’t in the middle of a lake at which point I would have clipped out the lakes).
I welcome any criticism of my methodology and I’ve purposefully went into a minutia of detail so as to encourage constructive criticism.
I have been watching a very interesting video on the FOSS4G NA’s YouTube channel by Steve Bennett a software engineer from a company called Data 61 all about their creation and implementation of an opensource website for all geospatial data in Australia. I wasn’t aware of this website and decided to take a look at it, it’s called http://nationalmap.gov.au/. When the interface loads uses Bing as its default basemap.
It has a very impressive range of datasets available to view. A very useful feature is that where available you can download the data directly. The National Map does not host any data directly but ‘provides a federated interface to the diverse servers already operated by the various government agencies, running Geoserver, THREDDS Data Server, ArcGIS Server, CKAN, Socrata, and many more’.
To create the National Map they created a new library called ‘TerriaJS’ that makes it straightforward to build 3D web-based geospatial data explorers. The crucial point to make with TerriaJS is that it is entirely opensource and is already being used by a host of different companies both inside and outside Australia. The road that Steve and his colleagues have traversed has been long with a screenshot of his YouTube video shown below to give the timeline.
Screenshot from FOSS4G NA’ YouTube video
The search function works very well and is very responsive. I searched for and downloaded the Australian post code areas, it worked flawlessly. A screenshot of querying the post code layer is shown below.
Querying the post code layer
Any query can be downloaded as either a CSV or JSON. One post code caught my attention for the sheer size of it at 1,203,979km². To put that in perspective that one post code is over twice the size of France or over 17 times the size of the Republic of Ireland.
The website has the ability to upload data. I uploaded a JSON file of Irish speed camera zones, it worked flawlessly as shown below.
In anticipation of results of the 2016 Irish census I thought that it would be interesting to calculate the mean centre of population from the 2011 census. The data was downloaded from the Central Statistics Office website. The next step was to download the census boundaries from here. Once downloaded they were opened in ArcMap 10.2.2 and the Mean Center (sic) tool was used from the Spatial Statistics toolbox to calculate the mean centre with the total population for each small area used for the weight. Below is the mean population centre for the Republic, it is located in the townland of Capard, Co. Laois.
The first step was to download a Sentinel-1 image from the Copernicus website. I downloaded an image captured on the 4th April 2016.
This is the download dialogue box on the ESA Copernicus website.
The next step was to open the image in the SNAP toolbox. Once it was opened, the next step was to calibrate the image. You have to go to Radar –> Radiometric –> Calibrate as shown below.
Calibrate Menu Option
To calibrate you have to select both bands VH and VV on the second tab that is displayed after you click calibrate and the parameter you need it Sigma0.
Once the calibration has run then a second product appears in the window.
Extracting Information on Ships:
The next step was to extract information on the ships. The option is located at:
Go to Radar –> Feature Extraction –> Ocean Tools –> Ocean Object Detection
You have to make sure that the calibrated image is the one selected as shown.
The Sigma0_VH is the band to select. You then click run. It is quite memory and CPU intensive and took my computer 35 minutes to complete (it’s only a few months old with a lot of RAM so that may have helped). It is a very intensive process as you can see below.
Task Manager for Object Detection
The end result in the SNAP toolbox was as follows:
Ship Detection Results
Individual Ships looks as follows:
Individual Ships in Results Window
As Anne has desribed on her blog, the next step was to located the log file (xml file) and import it into Excel and save as a CSV as shown below.
CSV File from Excel
This was then imported into QGIS and a simple map was created with OSM as the background. The end result is shown below: