Running Selenium and Headless Chrome on AWS Lambda Layers

Selenium and Headless Chrome on AWS Lambda Layers AWS has extended the timeout limit for Lambda functions from 5 to 15 minutes, also AWS released new Lambda layers feature at re:Invent 2018, with these new features, we can now move Selenium tests to server-less frameworks without any performance issues! After many attempts of using different version of chrome drivers and binaries — I eventually find a way to get it work — ChromeDriver was able to run and interact with Headless Chrome inside a Lambda Layer. I created (≥1.34.0) project to publish and use Lambda Layers with Selenium and Headless Chrome, thus team is able to do UI test using Python without running Selenium on server or local machine. Serverless Framework Selenium and Headless Chrome Incompatible versions of serverless-chrome, chromedriver, and Selenium can cause error. These are the versions that play well together in through my testing eventually: Chrome not reachable Python3.6 Selenium2.37 ChromeDriver2.37 Serverless Chrome v1.0.0.41 File Structure There are 2 sub sls projects in root directory, is Selenium Lambda Layer which stores selenium libs, Headless Chromium driver and binary. is normal lambda function to do the UI Testing seleniumLayer lambda root ── /seleniumLayer/ # lambda layers ├── /selenium lambda layer of selenium lib │ └──/python/ # python libs │ └── /lib/ │ └── /python3.6/* ├── /chromedriver/ # lambda layer of headless Chrome │ ├── /chromedriver # chrome driver │ └── /headless-chromium # headless chrome binary └── /serverless.yaml ── /lambda/ # lambda function ├── /handler.py # source code of lambda function └── /serverless.yaml # serverless config Install Selenium library Lambda runtimes include paths in the /opt directory to ensure that your function code has access to libraries that are included in layers. To include libraries in a layer, place them in python/lib/python3.6/site-packages/ # download Selenium 2.37 to layer directory $ pip3.6 install -t seleniumLayer/selenium/python/lib/python3.6/site-packages selenium==2.37 Install Headless Chrome Driver and Binary Go to directory, install chrome binary and driver root # download chrome driver $ mkdir -p seleniumLayer/chromedriver $ cd seleniumLayer/chromedriver $ curl -SL https://chromedriver.storage.googleapis.com/2.37/chromedriver_linux64.zip > chromedriver.zip $ unzip chromedriver.zip $ rm chromedriver.zip # download chrome binary $ curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-41/stable-headless-chromium-amazonlinux-2017-03.zip > headless-chromium.zip $ unzip headless-chromium.zip $ rm headless-chromium.zip Copy following code to /seleniumLayer/serverless.yaml service: selenium-layer provider: name: aws runtime: python3.6 region: ap-southeast-2 timeout: 900 layers: selenium: path: selenium CompatibleRuntimes: [ "python3.6" ] chromedriver: path: chromedriver description: chrome driver layer CompatibleRuntimes: [ "python3.6" ] resources: Outputs: SeleniumLayerExport: Value: Ref: SeleniumLambdaLayer Export: Name: SeleniumLambdaLayer ChromedriverLayerExport: Value: Ref: ChromedriverLambdaLayer Export: Name: ChromedriverLambdaLayer Lambda Function and Copy following code to /lambda/handler.py from selenium import webdriver from selenium.webdriver.chrome.options import Options def hello(event, context): options = Options() options.binary_location = '/opt/headless-chromium' options.add_argument('--headless') options.add_argument('--no-sandbox') options.add_argument('--single-process') options.add_argument('--disable-dev-shm-usage') driver = webdriver.Chrome('/opt/chromedriver',chrome_options=options) driver.get('https://www.neaminational.org.au/') body = f"Headless Chrome Initialized, Page title: {driver.title}" driver.close(); driver.quit(); response = { "statusCode": 200, "body": body } return response Copy following code to /lambda/serverless.yaml service: selenium-lambda provider: name: aws runtime: python3.6 region: ap-southeast-2 timeout: 900 functions: hello: handler: handler.hello layers: - ${cf:selenium-layer-dev.SeleniumLayerExport} - ${cf:selenium-layer-dev.ChromedriverLayerExport} Deploy Lambda Layers Go to / directory seleniumLayer $ sls deploy Deploy Lambda Function Go to / directory lambda $ sls deploy Start Testing Go to / directory lambda $ sls invoke --function hello You should get Response as below { "statusCode": 200, "body": "Headless Chrome Initialized, Page title Home | Neami National" } I hope you have found this article useful, You can find complete project in my : GitHub repo yai333/Selenium-UI-testing-with-AWS-Lambda-Layers Learn more How To Add NodeJs Library Dependencies in a AWS Lambda Layer With Serverless Framework