Authorized requests to S3 bucket

Protected S3 buckets, protected files

This notebook shows the finished product of adding basic permissioning to an S3 bucket

We use basic auth which is an HTTP protocol for simple auth on web-accessible files. https://en.wikipedia.org/wiki/Basic_access_authentication

Basic auth isn’t very secure — however, we pair this with HTTPS and restrict access to the s3 bucket.

Set up some python stuff

In [1]:

import requests; import json

Access secure endpoint without auth

first were gonna try to access this file without any credentials

In [2]:

url = 'https://d17nii79zr8aom.cloudfront.net/success.json'resp = requests.get(url)resp.content

Out[2]:

'Unauthorized'

Next we add basic auth params

Access secure endpoint with auth!

In [3]:

user, password = 'user', 'pass'resp = requests.get(url, auth=(user, password))data = json.loads(resp.content)print json.dumps(data, indent=4)

Out [3]:

{"status": "success","secret": "yay now we can lockdown s3 files!"}

Okay cool, hackers dont care about the front door. Lets try to acess the direct url of the S3 object

Direct S3 bucket access

In [4]:

direct_url = 'https://s3.amazonaws.com/locked-box/success.json'resp = requests.get(direct_url)print resp.content

Out [4]:

<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>58277072A5A1F927</RequestId><HostId>2CmgTzauvXbV0+bf9jMKvlXj3ViMNw4bUL1JMnu4L1QqHfOu0/eHJfG0cxunR0nq7hrVJb8HpQ0=</HostId></Error>

okay obviously that didnt work — we didnt even use the credentials. Lets pretend we know the login credentials but use them directly on the S3 bucket and not the secure endpoint.

In [5]:

user, password = 'user', 'pass'resp = requests.get(direct_url, auth=(user, password))print resp.content

Out [5]:

<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><Message>Unsupported Authorization Type</Message><ArgumentName>Authorization</ArgumentName><ArgumentValue>Basic dXNlcjpwYXNz</ArgumentValue><RequestId>97760837E823C675</RequestId><HostId>MaKcLnOik5Bq4zV+2v9fNzKqikz7JEHdEIv7TJYUP+67jJmdU4w9ekOr9jaZIbGHj+Wz68M4RcI=</HostId></Error>

that didnt access it! woooo!

success 🤘🏽

We can lock down S3 files with a Lambda function for auth — and a Cloudfront HTTPS endpoint as an acess point