1,339 reads

Visualizing Asteroids in Python with Bokeh and NASA APIs

by Ashley WansOctober 3rd, 2020

Too Long; Didn't Read

Astronomy Picture of the Day (APOD) is one of many APIs available through NASA Open APIs. The Near Earth Object Webservice (NeoWs) API is a RESTful web service for near earth Asteroids. We'll explore a small portion of all the available information provided through these APIs. We’ll visualize that data with a Python library called Bokeh to help contextualize it for the viewer. Check out the interactive visualization created in this tutorial or see the code on GitHub.

Coins Mentioned

featured image - Visualizing Asteroids in Python with Bokeh and NASA APIs

A GET request to the Astronomy Picture of the Day (APOD) web service on September 18, 2020, returned the above photo of the spiral galaxy Arp 78. APOD is one of many APIs available through NASA Open APIs. Like APOD, some include imagery, such as the Earth Polychromatic Imaging Camera (EPIC) API, which returns images like this:

Others include data. The Coronal Mass Ejection Analysis API, which is part of the DONKI suite of space weather forecasting tools, returns data that looks like this:

[
   {
      "time21_5":"2016-09-06T14:18Z",
      "latitude":-20.0,
      "longitude":120.0,
      "halfAngle":31.0,
      "speed":674.0,
      "type":"C",
      "isMostAccurate":true,
      "associatedCMEID":"2016-09-06T08:54:00-CME-001",
      "note":"",
      "catalog":"SWRC_CATALOG",
      "link":"https://kauai.ccmc.gsfc.nasa.gov/DONKI/view/CMEAnalysis/11233/-1"
   },
   {
      "time21_5":"2016-09-15T04:24Z",
      "latitude":-18.0,
      "longitude":-122.0,
      "halfAngle":43.0,
      "speed":722.0,
      "type":"C",
      "isMostAccurate":true,
      "associatedCMEID":"2016-09-14T23:36:00-CME-001",
      "note":"Measured with swpc_cat using C3 and STA Cor2 imagery.",
      "catalog":"SWRC_CATALOG",
      "link":"https://kauai.ccmc.gsfc.nasa.gov/DONKI/view/CMEAnalysis/11256/-1"
   }
]

In this article, we'll dig into an API that, like the one above, returns NASA data in JSON format. We’ll explore a small portion of all the available information provided through NASA Open APIs. We'll visualize that data with a Python library called Bokeh to help contextualize it for the viewer.

Check out the interactive visualization created in this tutorial or see the code on GitHub.

The NeoWs API

The Near Earth Object Webservice (NeoWs) API caught my attention immediately, because... well, who wouldn't want to know about asteroids hurtling towards earth at this very moment? The NASA Open APIs documentation describes it this way:

"NeoWs (Near Earth Object Web Service) is a RESTful web service for near earth Asteroid information. With NeoWs a user can: search for Asteroids based on their closest approach date to Earth, lookup a specific Asteroid with its NASA JPL small body id, as well as browse the overall data-set."

NeoWs data is also relatively accessible for someone like myself with no subject matter knowledge. Unlike some of the space weather APIs, which I find mostly unintelligible with my lack of space weather training, NeoWs returns data with self-explanatory keys like "estimated_diameter" and "miss_distance":

{
   "links":{
      "next":"http://www.neowsapp.com/rest/v1/feed?start_date=2020-09-30&end_date=2020-09-30&detailed=false&api_key=DEMO_KEY",
      "prev":"http://www.neowsapp.com/rest/v1/feed?start_date=2020-09-28&end_date=2020-09-28&detailed=false&api_key=DEMO_KEY",
      "self":"http://www.neowsapp.com/rest/v1/feed?start_date=2020-09-29&end_date=2020-09-29&detailed=false&api_key=DEMO_KEY"
   },
   "element_count":12,
   "near_earth_objects":{
      "2020-09-29":[
         {
            "links":{
               "self":"http://www.neowsapp.com/rest/v1/neo/54054531?api_key=DEMO_KEY"
            },
            "id":"54054531",
            "neo_reference_id":"54054531",
            "name":"(2020 SN2)",
            "nasa_jpl_url":"http://ssd.jpl.nasa.gov/sbdb.cgi?sstr=54054531",
            "absolute_magnitude_h":24.431,
            "estimated_diameter":{
               "kilometers":{
                  "estimated_diameter_min":0.0345425963,
                  "estimated_diameter_max":0.0772395934
               },
               "meters":{
                  "estimated_diameter_min":34.5425962673,
                  "estimated_diameter_max":77.239593373
               },
               "miles":{
                  "estimated_diameter_min":0.0214637676,
                  "estimated_diameter_max":0.0479944434
               },
               "feet":{
                  "estimated_diameter_min":113.3287315376,
                  "estimated_diameter_max":253.4107475218
               }
            },
            "is_potentially_hazardous_asteroid":false,
            "close_approach_data":[
               {
                  "close_approach_date":"2020-09-29",
                  "close_approach_date_full":"2020-Sep-29 13:21",
                  "epoch_date_close_approach":1601385660000,
                  "relative_velocity":{
                     "kilometers_per_second":"8.7170431155",
                     "kilometers_per_hour":"31381.355215948",
                     "miles_per_hour":"19499.1537451539"
                  },
                  "miss_distance":{
                     "astronomical":"0.1257314283",
                     "lunar":"48.9095256087",
                     "kilometers":"18809153.865737721",
                     "miles":"11687466.2571577098"
                  },
                  "orbiting_body":"Earth"
               }
            ],
            "is_sentry_object":false
         },

...

It also contains the eminently important "is_potentially_hazardous_asteroid" key. I didn't include this information in the visualization in this article because I optimistically assume this key will almost always be set to false.

NeoWs data is retrieved with a simple GET request:

GET https://api.nasa.gov/neo/rest/v1/feed?start_date=START_DATE&end_date=END_DATE&api_key=API_KEY

Visualizing Asteroids

Although it's clear what fields like "miss_distance" mean, the JSON data alone doesn't give us much sense of perspective. This is where our Bokeh visualization comes in.

We will create a chart that looks like this one created on September 27th:

This plot contains circles representing asteroids that reached their closest distance to Earth on the day the code was run (in this case, on September 27th). The time the asteroid approaches Earth is in UTC along the X-axis and miss distance in miles is along the Y-axis.

The relative size of each asteroid is given by the radius of each circle. The size of all asteroids on our plot is scaled up by a factor of a million. This is so that they are visible despite the Y-axis being in millions of miles and the largest asteroid on the plot being less than a mile across.

The moon is plotted in blue in the lower left-hand corner of the chart. This is to give the viewer a sense of how far the asteroids really are from Earth. The moon's radius is not to scale relative to the asteroids, as it is much larger than the largest asteroid on the chart and, if scaled up by a factor of a million, would be larger than the chart itself.

Although this plot is a static HTML page, it was created entirely in Python. Bokeh is an excellent visualization library for creating interactive plots with a limited amount of code. I chose Bokeh for this visualization because I've worked with it several times and found development in Bokeh to be extremely fast relative to the quality of output it produces.

Querying NeoWs

Before you can start querying NASA Open APIs, you will need to create an API key. This is fast and free. Instructions on how to do this can be found at the top of api.nasa.gov.

We'll use the requests library to make GET requests against the NeoWs API. First, make sure requests is installed on your system:

pip install requests

Making the GET request in Python is simple:

        params = {'start_date' : today_str,
                'end_date' : today_str,
                'api_key' : self.api_key}
        
        get_request = requests.get(url = self.api_endpoint, params = params)
        json_resp = get_request.json()

json_resp

now contains a JSON blob from NeoWs like the one above.

We only need a few items from

json_resp

to create our plot, so I extract that information before passing the list of asteroids back to the caller. In this case, we only want the asteroid's name, close approach time, miss distance, diameter, and it's JPL URL. We will append these values as a tuple to our list of asteroids.

miss_miles = neo['close_approach_data'][0]['miss_distance']['miles']
diameter_miles = neo['estimated_diameter']['miles']['estimated_diameter_max']
url = neo['nasa_jpl_url']
asteroid_list.append((name, close_approach_epoch, miss_miles, diameter_miles, url))

Note that the approach date field is in BDT time. I use Astropy to convert it to UTC time. Install Astropy:

pip install astropy

And then convert BDT to UTC before appending the data to our list:

close_approach_epoch = datetime.datetime.strptime(neo['close_approach_data'][0]['close_approach_date_full'], '%Y-%b-%d %H:%M')
close_approach_epoch = Time(close_approach_epoch, scale='tdb')
close_approach_epoch = str(close_approach_epoch.tt)

Plotting with Bokeh

Now that we have our list of tuples of asteroid data, we can create our visualization. First install Bokeh:

pip install bokeh

Bokeh has a lot of great examples available. Even after working with Bokeh a few times, I still find it helpful to skim through the examples to find the style of plot I would like to create. I then use that code as a starting point.

In this case, I started with the color scatter example and modified from there. The final code looks quite a bit different, but I do think the example code saved time that would otherwise have been spent Googling things I've forgotten.

Our plotting code begins by setting up our data sources:

    def plot_asteroids(self, neos_list:list):
        # We will label each circle on our plot with the NEO name and it's diameter
        names = [(name + " {:.2f}".format(dia) + " M wide") for (name, miss_time, miss_dist, dia, url) in neos_list]
        names.append('Moon') # We will add the moon to our plot to help give a sense of scale

        # Drop the nanoseconds we don't need off the end of the timestamps
        x = [datetime.datetime.strptime(miss_time[0:-7], '%Y-%m-%d %H:%M:%S') for (name, miss_time, miss_dist, dia, url) in neos_list]
        x.append(datetime.datetime.strptime(self.date_str + ' 04:00', '%Y-%m-%d %H:%M')) # We'll put the moon at 4 hours

        # Distance from Earth will go along the Y axis
        y = [float(miss_dist) for (name, miss_time, miss_dist, dia, url) in neos_list]
        y.append(238900)  # The moon is 238,900 from Earth

        # Let's represent each NEO's relative size with the diameter of each circle
        # Since most NEOs are less than a mile wide, we will need to scale them by a factor of 1e7
        # to see them on our plot
        sizes = [float(dia) for (name, miss_time, miss_dist, dia, url) in neos_list]
        radii = [size * 1e7 for size in sizes]
        radii.append(1e6) # This is an arbitrary size, since the moon is actually much larger than any of our NEOs

        # Users will be able to click on each circle to view the NASA's page for each NEO
        urls = [url for (name, miss_time, miss_dist, dia, url) in neos_list]
        urls.append('https://moon.nasa.gov/')  # Moon website!

        # Let's make the NEOs coming close to Earth red and the
        # ones farthest from Earth green
        colors = []
        min_dist = min(y)
        max_dist = max(y)
        range_dist = max_dist - min_dist
        for (name, dist) in zip(names, y):
            if name == 'Moon':
                colors.append('#0EBFE9')  # Let's make the moon blue so that 
                                          # it's clear it's different from the NEOs
            elif dist == min_dist:
                colors.append("#%02x%02x%02x" % (255, 0, 0))
                
            elif dist == max_dist:
                colors.append("#%02x%02x%02x" % (0, 255, 0))

            else:
                perc = (dist - min_dist)/range_dist
                num = int(perc * 500)
                if num > 255:
                    colors.append("#%02x%02x%02x" % (0, num-255, 0))
                else:
                    colors.append("#%02x%02x%02x" % (255-num, 0, 0))

        # Create the ColumnDataSource that we will use for plotting and creating labels
        source = ColumnDataSource(data=dict(hours=x,
                            distance=y,
                            names=names,
                            radii=radii,
                            colors=colors,
                            url=urls))

The

colors

list provides the color for each of our circles. The asteroids with smaller miss distances are shades of red and the ones with further miss distances are shades of green. Note that this is all relative to each other; the closest asteroid in the list is set to red regardless of its absolute miss distance.

Bokeh does provide built-in pallettes for colorizing plots. I found that in my case the auto-magic behind the pallettes colorized my circles by radius rather than Y-axis value, so I constructed the colors list myself to achieve the effect I wanted.

After constructing each list, I wrapped all the data in a

ColumnDataSource

. It's not required to do this; you can pass the data to Bokeh directly for plotting. However, the

ColumnDataSource

provides data mapping that drives a lot of Bokeh's features. In this case, the ColumnDataSource allows us to construct all the labels for our circles with a single call:

labels = LabelSet(x='hours', y='distance', text='names', source=source, text_font_size='8pt')

source=source

refers to our ColumnDataSource variable

source

defined above.

It also allows us to set up a TapTool that will open each asteroid's JPL link when a user clicks on it:

# Add TapTool call back that will open a new tab with our NEO URL on a click
taptool = p.select(type=TapTool)
taptool.callback = OpenURL(url="@url")

"@url"

refers to the "url" field in our ColumnDataSource.

Bokeh allows us to add some annotations on our chart. I used this feature to credit the source of our data and make an explanatory note about the moon:

# Let's label each NEO to make our plot more meaningful
citation = Label(x=850, y=70, x_units='screen', y_units='screen',
                        text='Data From NASA NeoW API', render_mode='css',
                        border_line_color='black', border_line_alpha=0.6,
                        background_fill_color='white', background_fill_alpha=0.6)

# Let's make a note comparing the moon's real diameter to the largest NEO on this plot
max_size = max(sizes)
scale = " {:.2f}".format(2158.8/max_size) # Moon is 2,158.8 miles in diameter

moon_note = Label(x=150, y=30, x_units='screen', y_units='screen', text='Moon diameter not to scale; it is ' + scale + 'x larger than largest NEO on this plot.', render_mode='css',
                border_line_color='black', border_line_alpha=0.6,
                background_fill_color='white', background_fill_alpha=0.6, text_font_size="8pt")

With these components in place, we can construct our output file and display our plot.

# Output our plot to a static HTML file
output_file("neos_today.html", title="NEOs Demo", mode="cdn")

# Set up our plots
TOOLS = "pan,wheel_zoom,box_zoom,reset,box_select,lasso_select,tap"
p = figure(tools=TOOLS, x_axis_type="datetime", plot_width=1250, height=650, x_axis_label="Hours After Midnight", 
    y_axis_label="Miles From Earth", title=self.plot_title)
p.circle(x="hours", y="distance", source=source, radius="radii", fill_color="colors", fill_alpha=0.6, line_color=None)

labels = LabelSet(x='hours', y='distance', text='names', source=source, text_font_size='8pt')

p.add_layout(labels)
p.add_layout(citation)
p.add_layout(moon_note)

# Add TapTool call back that will open a new tab with our NEO URL on a click
taptool = p.select(type=TapTool)
taptool.callback = OpenURL(url="@url")

# Display the plot in a browser
show(p)

output_file

defines our static HTML output file. The

show

call will display this file in a browser and can be omitted if you don't wish to display your plot immediately.

TOOLS

refers to various interactive user tools Bokeh provides, such as zooming and panning tools. They appear to the left of my plot (the display location of tools can be adjusted if you wish) and allow users to explore the plot by adjusting it in various ways.

Sharing your visualization

Our output is a static HTML file. I'm using a public S3 bucket to make my example chart available online. I could also include it on another website, or share it via email.

Taking this example further

Bokeh does provide a server im plementation for building apps that involve periodic updates to plotted data. I've used Bokeh Server several times and found it easy to use, but not appropriate for embedding Bokeh into other more complex applications.

Nonetheless, an interesting extension of this project would be to build a simple app that updates asteroid data daily and posts updated plots to a website or S3 bucket. I also wondering about creating a version of this plot that doesn't scale the size of the asteroids up, but instead uses animation to zoom in on each asteroid, providing a better sense of scale in regards to the asteroids' true sizes.

Final thoughts

This project is a starting point for further exploration into available NASA APIs. Plotting data with Bokeh is the easy part; the limiting factor for most of us will be understanding the NASA data itself. I look forward to putting in a little extra research on some of the more accessible APIs to see what I can learn and build!