With the advent of Cloudflare’s new , it is now easy to download and inspect a zone’s access logs by a single RayID (Cloudflare’s fancy term for a request ID), a defined time period, or from the last downloaded. In our case, we ship all of our other logs to so everyone can analyze and parse them in any way imaginable! Enterprise Log Share (ELS) REST API RayID Sumo Logic Unfortunately, there seems to be no cut-and-dry way to integrate the two. Thankfully, Lambda functions are now able to be run on a scheduled interval using — sort of like you would a cron job on a server. AWS CloudWatch Scheduled Events There are a couple of clear benefits of using a Lambda function instead of a job on a physical/virtual server you run yourself: Execution of a Lambda function , and it may actually cost nothing if you remain within the free tier AWS hands out every month Low cost: costs next to nothing In my case, I used the to allow me to quickly get up and running, invoke a function locally (which is key to developing and iterating quickly), and it makes deployments simple. I also recommend checking out , too. Ease of development: Serverless framework apex Lambda functions are This is important for things which require speed, such as parsing and ingesting logs for a high-traffic site like ours. Speed: fast. Write the code, test and deploy — that’s it! No Chef, no Docker, no nothing… just write your function, ship it and it is live. No servers to build/maintain: How It Works First, you’re going to want to to ingest the Cloudflare logs. Set the source category, host, etc. at SumoLogic to make the configuration of the Lambda function easier. create a new HTTP source Next, create a new Serverless function: $ mkdir -p ~/cloudflareToSumoLogic $ cd ~/cloudflareToSumoLogic $ serverless create \--name cloudflareToSumoLogic \--template aws-nodejs \--path ~/cloudflareToSumoLogic Let’s walk through the configuration file: serverless.yml functions: cloudflareToSumoLogic: # This is the name of the Lambda function **_\# This is the name of the handler; this is the "main"_ \# _function called by Lambda_** handler: index.handler **_\# This function was written to be run every one minute # Any adjustments will require code changes..._** events: - schedule: rate(1 minute) **_\# Required environment variables:_** environment: **\# Grab this URL from the Sumo Logic source you just created** SUMO\_ENDPOINT: ... **\# Get these values from the Cloudflare admin UI** CLOUDFLARE\_ZONE\_ID: ... CLOUDFLARE\_AUTH\_EMAIL: ... CLOUDFLARE\_AUTH\_KEY: ... Finally, you’re going to need the function: Now, try it out! $ serverless invoke local -f cloudflareToSumoLogic startTime: 2017-02-11T15:07:00.000ZendTime: 2017-02-11T15:08:00.000Z cloudflareOpts: { method: 'GET',hostname: 'api.cloudflare.com',path: '/client/v4/zones/.../logs/requests?start=1486825620&end=1486825680',headers:{ 'X-Auth-Email': ' ..','X-Auth-Key': '...' } } . res.statusCode: 200 res.headers: { date: 'Sat, 11 Feb 2017 15:38:40 GMT','content-type': 'application/json','transfer-encoding': 'chunked',connection: 'close','set-cookie': [ '__cfduid=...; expires=Sun, 11-Feb-18 15:38:39 GMT; path=/; domain=.cloudflare.com; HttpOnly' ],vary: 'Accept-Encoding','x-cf-request-id': '00000000-0000-0000-0000-000000000000','strict-transport-security': 'max-age=31536000','served-in-seconds': '0.355',server: 'cloudflare-nginx','cf-ray': '...-ORD' } Log events: 659 Everything look good? It’s time to deploy: $ serverless deploy Serverless: Packaging service...Serverless: Uploading CloudFormation file to S3...Serverless: Uploading service .zip file to S3 (9.45 KB)...Serverless: Updating Stack...Serverless: Checking Stack update progress........................Serverless: Stack update finished...Serverless: Removing old service versions... Service Informationservice: cloudflareToSumoLogicregion: us-east-1api keys:Noneendpoints:Nonefunctions:cloudflareToSumoLogic Analyzing Ingested Logs in Sumo Logic Now that your logs are being ingested every minute, you can run the following query and you should be able to start seeing results like the screenshot above: _sourceCategory=...| json auto| toLong(num(timestamp)) as timestamp| timestamp as _messageTime| fields - timestamp| fields - _raw Replace with the HTTP source category that you created earlier for your Cloudflare logs. This snippet parses the JSON lines automatically, translates the log timestamp to a time object, replaces the with the log timestamp, then remove the raw JSON object and its ingest timestamp. _sourceCategory timestamp _messageTime
