Using AWS DynamoDB with Flask

In this tutorial I will go over how to easily integrate AWS DynamoDB with Flask, Python’s web development server. You will be creating a simple RESTful API that allows you to pull items from DynamoDB.

DynamoDB

DynamoDB is AWS’ answer to the NoSQL cloud database. Similar to other AWS services, DynamoDB allows you to spin up databases for an application without having to worry about infrastructure, backups, and redundancy. You are able to interact with the database through the AWS console, or through the AWS API. In this tutorial we will be interacting with AWS using their command line package, as well as through boto3, a Python wrapper. Both of these methods make use of the AWS API.

Boto3

Boto3 is a Python package that acts as a wrapper around the AWS CLI. It is the recommended package for interacting with AWS using Python.

Flask

Flask is Python web framework. At the most basic level it provides a development server and debugger. Flask is highly extendable. Allowing the addition of other application features, such as: form validation, upload handling, and authentication. It will be used in this tutorial for it’s web server capabilities.

Prerequisites

This tutorial assumes that you have a basic understanding of the following:

• Linux CLI (bash)
• Python3 programming
• pip
• Flask
• AWS and IAM

If you do not have Python3 already installed on your local machine please download the latest version here.

pip, pythons package manager, should come with the Python installation. If for some reason it's not on your local machine go here for instructions on how to install it.

If you do not have the AWS CLI installed run the following terminal command:

pip install awscli --upgrade --user

awscli allows you to run AWS commands from the command line. With this package we are able to create EC2 instance, insert objects into an S3 instance, and delete items from DynamoDB tables, just to name a few examples. It's also a boto3 dependency allowing it to interact with AWS.

Next you need to configure awscli so that it has access and permissions to your AWS account.

aws configure

This should give you the following prompts:

AWS Access Key ID [None]: ***PASTE ACCESS KEY ID HERE*** AWS Secret Access Key [None]: ***PASTE SECRET ACCESS KEY HERE*** Default region name [None]: ***TYPE YOUR PREFERRED REGION*** Default output format [None]: json

The Access Key ID and Secret Access Key you enter here will define what kind of permissions awscli will have when interacting with AWS. You will get this information from the IAM console. If you have no previous experience with AWS permissions please go here for more information.

You should also specify the default region. This will be the region on which your awscli will run most of it's commands by default. A list of AWS available regions can be found here.

The default output format is set to json here in order to make it readable, but this can be changed.

Creating a Test Table

Since the only operation our API will expose is a very basic GET, you need to create a new DynamoDB table and populate it with a few items. This will be done using the AWS CLI.

First, create a json file that specifies the table schema.

Create a create-table.json file through the terminal.

touch create-table.json

And add the following code.

{"TableName": "YourTestTable","KeySchema": [{ "AttributeName": "Artist", "KeyType": "HASH" },{ "AttributeName": "SongTitle", "KeyType": "RANGE" }],"AttributeDefinitions": [{ "AttributeName": "Artist", "AttributeType": "S" },{ "AttributeName": "SongTitle", "AttributeType": "S" }],"ProvisionedThroughput": {"ReadCapacityUnits": 5,"WriteCapacityUnits": 5}}

This will tell AWS to create a new table named YourTestTable with string attributes Artist and SongTitle. It also specifies that both the Artist and SongTitle attributes will be used as the primary key.

The ProvisinedThroughput key tells DynamoDB what to set the read and write capacities to. More information can be found here

Next, create a file named batch-write.json.

touch batch-write.json

And insert the json code below:

{"YourTestTable": [{"PutRequest": {"Item":{"Artist": {"S": "Bowie"},"SongTitle": {"S": "Life on Mars"}}}},{"PutRequest": {"Item": {"Artist": {"S": "Rolling Stones"},"SongTitle": {"S": "Paint it Black"}}}},{"PutRequest": {"Item" : {"Artist": {"S": "The Doors"},"SongTitle": {"S": "Light My Fire"}}}}]}

The json file above will be parsed by awscli when running a batch write command on our table. As you can see we are passing a list of three PutRequest, which each contains an Item that consist of a dictionary of key/value pairs corresponding to our table attributes.

Finally, we run the terminal commands to create and populate our table.

aws dynamodb create-table --cli-input-json file://create-table.json 

aws dynamodb batch-write-item --request-items file://batch-write.json

Should no error have occurred you should now have a new table that is populated. You can go to the AWS console to verify that the table is there.

Note: The table will be created in the default region that was specified as part of the AWS configuration.

Project Setup

Now that your local machine has all of the dependencies installed you now need to create the root project directory.

mkdir flaskdynamodb cd flaskdynamodb

Since we will be installing python packages we need to create a virtual environment. Creating a virtual environment copies the Python executable to the package so that any calls to python from the command line will run the virtual environments local Python executable. The virtual environment will also install pip packages to this same environment. Preventing package conflicts with the system-wide packages.

Note: In some linux distributions the python virtual environment module does not come pre-installed. In the case of Ubuntu and Debian distributions, you would need to install it with the following command **sudo apt-get install python3-venv**. If the commands below do not work on your system you might need to install **python3-venv**

Create the virtual environment with the following terminal commands:

python3 -m venv env source env/bin/activate

The -m option tells python to run the virtual environment module, and create a new virtual environment directory named env.

The source env/bin/activate command activates the virtual environment. This means that your system will now use the Python executable and pip packages installed within the virtual environment folder.

Note: In the python command above is assumed **python3** is referring to Python3. This command might not work if you have **python** linked to Python3 on your system

Installation

Now that the project directory and virtual environment are setup it’s time to install flask and boto

pip install flask pip install boto3

I will mention again that these packages are going to be installed to the virtual environment directory. If you were to look at the env/lib/site_packages directory you would see the directories for boto3 and flask.

Flask Configuration

Before any code is written you need to export the environment variable which tells flask what the main application file is going to be.

export FLASK_APP=app.py

As you might have noticed the file app.py has not been created yet. Later on we will create this file, which will contain our Flask application code.

Application Code

You will now create the files where the actual code is going to reside.

touch app.py aws_controller.py

Add the following code to the app.py:

from flask import Flask

app = Flask(__name__)

@app.route('/')def index():return "This is the main page."

@app.route('/get-items')def get_items():return 'TODO: Implement Functionality'

if __name__ == '__main__':app.run()

The above code creates a Flask app with two routes, / and /get-items, exposed. At the moment these routes will simply respond with text. We will return to this file later in order to have /get-items return our call to AWS.

Note: If you are not sure how Flask works take a look at their quickstart guide.

It’s time to implement the code that actually interacts with DynamoDB in the aws_controller.py file.

import boto3

dynamo_client = boto3.client('dynamodb')

def get_items():return dynamo_client.scan(TableName='YourTestTable')

So what does the code above do?

1. First the boto3 module is imported.
2. dynamo_client = boto3.client('dynamodb') creates a client that allows us to interact with the DynamoDB API.
3. The function get_items runs a scan of the table name provided. In this case we are scanning YourTestTable. In DynamoDB a scan returns the entire table. This is in contrast to a query operation, which searches a table based upon a primary key attribute value. In this case a scan is run in order to get the entire table.

It’s time to go back and modify the Flask code in order for it to run the get_items function.

Change the app.py file to contain the following code:

from flask import Flask, jsonifyimport aws_controller

app = Flask(__name__)

@app.route('/')def index():return "This is the main page."

@app.route('/get-items')def get_items():return jsonify(aws_controller.get_items())

if __name__ == '__main__':app.run()

There are two things we changed in this file

1. We imported the aws_controller.py. This will give us access to the get_items function.
2. The /get-items route now returns the result from the aws_controller.get_items function, but not before passing the result to jsonify() which converts the object into the json equivalent which the client-side browser is able to render.

Now that everything is setup you can start the Flask web server by running the following command in your terminal.

flask run

If Flask was able to bind to the correct socket the server should now be running and you should see the below output.

(env) flaskdynamodb$ flask run* Serving Flask app "app.py"* Environment: productionWARNING: Do not use the development server in a production environment.Use a production WSGI server instead.* Debug mode: off* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)127.0.0.1 - - [12/Dec/2018 22:51:21] "GET / HTTP/1.1" 404 -127.0.0.1 - - [12/Dec/2018 22:51:21] "GET /favicon.ico HTTP/1.1" 404 -127.0.0.1 - - [12/Dec/2018 22:51:26] "GET /get-items HTTP/1.1" 200 -127.0.0.1 - - [12/Dec/2018 22:51:26] "GET /favicon.ico HTTP/1.1" 404 -

Open your browser and go to the url specified in the output. This will be http://127.0.0.1:5000/ by default.

Now enter the url for /get-items. http://127.0.0.1:5000/get-items.

You should see a list of all the items from the table.

That’s it! You now have a simple RESTful API that interacts with DynamoDB.

Thanks for following along.

GitHub

Originally published at blog.albertoacuna.com on December 13, 2018.