A few weeks ago, I shared the story behind the creation of my dashboard. In case you are not familiar with it, you can go and . Fire Alert read about it now As I explained in my previous post, I work with satellite data from NASA, which is kindly provided through an open API. Two satellites, Aqua and Terra, are provisioned with heat-detecting cameras that are able to discern significant temperature differences at the ground level, hence isolating hotspots that are wildfires. The typical report from the satellites looks like this: most likely Field Value Latitude 36.79887 Longitude 12.01467 Time 18/01/2023 12:21 This essential raw data can't be directly translated on a map as-is. There are a few issues with it: the API endpoint does not retain any memory about the previous calls, and will inevitably return the same observations multiple times. I want to recognize these instances as known hotspots without triggering new alerts; the satellites passing over the Earth at different times will detect patches of hot ground in the whereabouts of known hotspots. Most likely, all the observations are about a single event that is still ongoing. I want to compare new data to previous observations; a hotspot could abruptly disappear from the satellite survey: maybe the fire stopped, but it is possible that dense smoke or clouds are concealing the real situation on the ground. Therefore, I want hotspots to have some stickiness and stay on the dashboard even if they are briefly not being reported. Analyzing these requirements, I realized that I needed an appropriate definition for a hotspot event that goes further than a single observation at a point in time. To this end, I personally designed a coarse heuristic: a hotspot event is determined by any number of observations happening within a 2km radius and within a 24-hour time window. Why the 2km figure? Well, under optimal conditions, the granularity of the source data is 1km, so I gave myself some slack and doubled that number to limit false positives. However, the Satellite error can be vastly greater (up to 5km) if a specific portion of the image is at the edges of the scan performed. It is possible to account for this deviation, but I chose to keep it simple for now. High-level workflow Keeping in mind the aforementioned premises, I am now able to properly explain how data ingestion in works. First of all, I call the satellite API hourly to get updated data over a specified area. I will get a report, with a certain number of observations. As mentioned above, I want to make sure that each record I am processing is truly unique: I combine the latitude, longitude, and time of the observation to derive a unique key and filter the occasional duplicate. Fire Alert At this point, I always take note of the new hotspot location. This is important for traceability, but also to gather historical data over time and be able to perform further analysis in the future. I believe that in autumn, after the wildfire season, I will have more significant observations to build more knowledge about this phenomenon. The last step of the workflow is to apply my heuristic. I look for all recent fires that were reported in a radius of 2km from the new observation: if I find anything, it means that most likely there still is an ongoing fire in that location and I have to update the existing hotspot. Otherwise, I will ring a bell and report the new hotspot event, both on the dashboard and in the dedicated Telegram channel. Fire Alert Working with spatial data Due to the nature of my application, I needed to correctly process, persist and confront spatial data. Considering that I work with rather small distances, the distance between two pairs of coordinates can be roughly computed the Earth was flat (the Equirectangular approximation). as if However, it would be rather cumbersome and inefficient to search for all points within a certain radius, as I do with my heuristic, and makes it difficult to extend the dashboard with new features such as searching for events in specific geographical areas like some national wildlife park or, let's say, over the country of Greece. In order words, I needed a Geographic Information System, or GIS. Fortunately, it is the case that Postgres supports a GIS module, unsurprisingly called PostGIS. The plugin enables new, interesting data types like (an entity living on a Cartesian plane) or (an entity best represented on the Earth's surface). For example, a hotspot event can be easily persisted as a record in the following table: GEOMETRY GEOGRAPHY https://gist.github.com/Ipanov7/7d1f95921c7631cbe46a5dba7342210e#file-hotspot-sql Using additional methods like and (to parse a string representation of a entity) it becomes trivial to select all hotspot events in the whereabouts of new observations: ST_Distance ST_GeogFromText GEOGRAPHY https://gist.github.com/Ipanov7/2ffcc8a7543fcf83cbe3d257d6dccc19#file-select-hotspot-distance-sql Setting up PostGIS requires specific settings. To enable it on my RDS instance, I followed these . instructions Conclusion In this post, I provided an under-the-hood overview of my work around my dashboard. I believe it gives the necessary clarity and context for the most curious users and about how it works. Fire Alert transparency If you are an expert in treating geospatial data and have any opinion or - even better - to share, please feel free to do in the comment or via email. Your suggestions will be highly appreciated! precious advice Last but not least, if you were intrigued by this article but never visited the dashboard, . actual Fire Alert you can remediate it at once now Also published here. Lead image source. References I Built a Tool to Track Wildfires: This Is Why Calculate distance, bearing and more between Latitude/Longitude points Managing spatial data with the PostGIS extension

Stop Teaching Ethics to Software Engineers

Managing the Spatial Data for My Wildfire Detection Dashboard

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

AI Is the Ultimate Philosopher: Here's Why

22 Best Tools to Use for Marketing Startups in 2022

15 Best Project Management Tools

5 Data Management Principles That Matter in 2021

5 Features To Consider When Looking For a Reliable Data Loss Prevention (DLP) Software To Buy

5 Software Documentation Tools That Make Managing Technical Documentation Easier

AI Is the Ultimate Philosopher: Here's Why

22 Best Tools to Use for Marketing Startups in 2022

15 Best Project Management Tools

5 Data Management Principles That Matter in 2021

5 Features To Consider When Looking For a Reliable Data Loss Prevention (DLP) Software To Buy

5 Software Documentation Tools That Make Managing Technical Documentation Easier

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps