🌎 Storing and Processing GIS data :
Storing and Processing GIS data has become really important nowadays as many organizations deal with geospatial data all the time. As you might know postgres offers a beautiful extension called Postgis that can be easily used to store, query and process spatial information. Pgrouting extends postgis to provide geospatial routing functionality.
Sometimes we have location data with respect to time. Such data is called spatiotemporal data and this data is very useful to track something or someone with respect to both space and time.
Trajectories in Postgis can be used to handle spatiotemporal data but if your data is huge then you could make use of Geomesa or Geospark with Apache Accumulo and Geoserver to handle your spatiotemporal Bigdata.
QGIS along with Timemanager plugin is a really cool open source way to visualize spatiotemporal data. (https://github.com/anitagraser). You could also check out Topi Tjukanov who has done some really cool stuff with qgis and timemanager. His articles and animations have helped a lot 🙏🏼🙏🏼
Special shoutout to mah boy Kepler.gl (https://github.com/uber/kepler.gl)
🌎 Pincode Mapping for India
India has post offices and each one has its own service area. The service areas encompassed by every post office have their own unique Pincode so that a post office knows its bounds. So historically we have divided our cities and strategically placed post offices that could service the entire city. This data is valuable. It would be really cool if we could extrapolate the pincode from a given latitude and longitude.
- Pincode Boundaries data for cities (https://github.com/datameet/PincodeBoundary)
- Pincode mapped to LatLongs (https://github.com/sanand0/pincode/blob/master/data/IN.csv)
- With the Pincode Boundaries dataset we can isolate the latlong to a specific city using a bounding box and check which pincode polygon contains the given latlong (Pretty straightforward) but this method would prove to be ambiguous for the latlongs located on the edges of the polygons (Literal edge cases…smh)
- With the Pincode latlong dataset we can write a spatial KDtree algorithm to check for the nearest latlong in the database to the given latlong and map it to the corresponding pincode. This method is not as accurate as we could have several cases where a latlong is closer to another pincode than it’s actual pincode
A simple function combining both algorithms into one
import geopandas as gpd from shapely.geometry import Point, Polygon import pandas as pd import numpy as np from scipy import spatial import pandas as pd
df = pd.read_csv('allindpincode.csv') latlongs = df.iloc[:,2:] d = np.array(latlongs) tree = spatial.KDTree(d)
data = gpd.read_file('allcitiescombined.geojson')
def getpincode(lat , long): lat = float(lat) long = float(long) p = Point(long,lat) for i in range(0,len(data)): if p.within(data['geometry'][i]) is True: pin = int(data['pin_code'][i]) else: pin = 0 if pin !=0: return pin else: latlongs = np.array([lat,long]) result = tree.query(latlongs) pin = int(data1.iloc[[result]]['postalcode']) return pin
I have written a simple python package encompassing these ideas to extrapolate pincode from a given latlong (FOR INDIAN CITIES)
pip install git+https://github.com/Sangarshanan/Pincode-Mapping.git
from geopincoder import pincode_mapper as pm
pincode = pm.geocode.to_pincode(28.7041, 77.1025)
🌎 Building a Routing Engine with Postgis and Pgrouting
Building a routing engine is pretty easy especially with OSM being an immensely useful openly available source of road network data.
Extract the .OSM file of any region you want to build a routing Engine for using osm extractor and selecting the bounding box
Now go to Postgres and create a new database or use existing
Create the extensions for postgis and pgrouting (install pgrouting first)
CREATE EXTENSION postgis; CREATE EXTENSION pgrouting;
We are using osm2pgrouting to easily import OpenStreetMap data into a pgRouting database (https://github.com/pgRouting/osm2pgrouting)
Follow the instructions in the github page to compile the tool(osm2pgrouting) by making use of boost, libpqxx, expat and cmake
osm2pgrouting — f Bangalore.osm — conf osm2pgrouting/mapconfig_for_cars.xml — dbname pgroute — username postgres — clean
The above given command takes the OSM file that I had extracted for Bangalore and configures the road network that allow cars. Now the road network is saved as Nodes and Edges in the pgroute database .
We can also perform incremental addition of data without using — clean
Now using the database with the road network data, it becomes quite easy to extrapolate the road path between two latlongs
- Connect to the database with the tables generated using OSM2pgrouting
- Take the two latlongs (Source and Destination) and find the nearest node to both those latlongs
- Now that we know the nodes, we have to find the path between these nodes. (Imagine the road network as a simple graph and our problem is to find the shortest path between two nodes in the graph)
- We can make use of any path finding algorithm (Dijkstra algorithm) to calculate the edges the path contains between the given nodes
The above attached gist gives us the function that can extrapolate the distance between two latlongs and gives you the answer in kilometers (Sorry America)
You could also check out the flask app that me and my buds built to do Pincode mapping and Routing
Although it’s much easier to use Google maps or other API’s to solve this problem building your very own routing engine is kinda fun
I recently also had the opportunity to use Graphhopper (https://github.com/graphhopper/graphhopper), which is a cooler open source alternative API to google maps
Setting up a routing engine with Graphhopper is super easy. Just clone the repo, copy the osm file there and run the shell script
git clone git://github.com/graphhopper/graphhopper.git cd graphhopper; git checkout master ./graphhopper.sh -a web -i bangalore.som
After running these you will see ‘Started server at HTTP 8989’, now go to http://localhost:8989/ and you should see a map of Bangalore. You should be able to click on the two points in the map and a route appears.
The below attached python script can help you save the road network as a gpx plot and plot the same in a Jupyter notebook using ipyleaflet
Hope this helps
Check out some of the useful links that I copy pasted from