Elevate Your Scraping Project With Puppeteer Extra

As highlighted in our guide to web scraping with Puppeteer, this browser automation library is a fantastic ally for extracting data from dynamic content sites. Still, like any other tool, it has its shortcomings. That's where Puppeteer Extra steps in! In this guide, we’ll introduce you to puppeteer-extra—a library that wraps puppeteer to extend it with plugin support. Get ready to take your Puppeteer scraping project to the next level! 🚀 What’s Puppeteer Extra? Puppeteer Extra is a lightweight wrapper around puppeteer that enables plugin integration through a clean interface. Although it's not developed by the team behind Puppeteer, this community-driven project has hundreds of thousands of weekly downloads and over 6k stars on GitHub 📈. Check out the GitHub stars chart below—it’s clear that the puppeteer-extra repo has been on a steady rise in popularity over the years: The plugins officially supported by Puppeteer Extra are: puppeteer-extra-plugin-stealth: To make it harder to detect Puppeteer as a bot. puppeteer-extra-plugin-recaptcha: To solve reCAPTCHAs and hCaptchas automatically. puppeteer-extra-plugin-adblocker: To reduce bandwidth and load times by applying a fast and efficient blocker for ads and trackers. puppeteer-extra-plugin-devtools: To make debugging the Puppeteer browser possible from anywhere. puppeteer-extra-plugin-repl: To make Puppeteer debugging and exploration easier and more enjoyable with an interactive REPL. puppeteer-extra-plugin-block-resources: To programmatically block resources like images, media files, CSS stylesheets, and more while loading pages. puppeteer-extra-plugin-flash: To allow Adobe Flash content to run on all sites without user interaction. puppeteer-extra-plugin-anonymize-ua: To anonymize the User-Agent header on all pages, with support for dynamic replacing. puppeteer-extra-plugin-user-preferences: To set custom Chrome/Chromium user preferences. On top of those, it integrates with the following community plugins: puppeteer-extra-plugin-minmax: To minimize and maximize the Puppeteer browser window in real time. puppeteer-extra-plugin-portal: To remotely view and interact with puppeteer sessions via the Chromium screencast API. Why Do We Even Need an Extra Version of Puppeteer? No doubt, Puppeteer is one of the top headless browser libraries for scraping and testing. But let’s be honest—it has its limits, especially when facing anti-bot tech like browser fingerprinting and CAPTCHAs. Read our guide to learn how to deal with reCAPTCHA automation. Websites armed with anti-bot defenses can easily detect and block Puppeteer scripts. If only there was a way to extend and customize Puppeteer’s default behavior... …well, that’s exactly what Puppeteer Extra is all about! Puppeteer Extra is like a power-up for Puppeteer, adding plugin support to tackle those major drawbacks. Instead of overriding or extending everything for you, it wraps Puppeteer and lets you register only the plugins you need. 🦸 puppeteer-extra: Setup and Plugins for Web Scraping You can add Puppeteer Extra to your project's npm dependencies with: npm install puppeteer-extra ⚠️ Note: puppeteer-extra requires puppeteer to work, so make sure both packages are installed in your project. Then, you have to import the puppeteer object from puppeteer-extra instead of the puppeteer library: const puppeteer = require("puppeteer-extra") // for ESM users: // const { puppeteer } from "puppeteer-extra" Everything in the Puppeteer API stays the same, but you get a little extra magic ✨. The puppeteer object now exposes a use() method to plug in Puppeteer Extra plugins. Time to dive into what these plugins can do, and see how they’ll level up your web scraping game! puppeteer-extra-plugin-stealth Puppeteer Extra Plugin Stealth, also known simply as Puppeteer Stealth, includes a set of configurations designed to reduce bot detection. It overrides Puppeteer’s detectable properties and settings that might expose it as a bot. For more details, check out our guide on how to avoid getting blocked with Puppeteer Stealth. ⚙️ Installation: npm install puppeteer-extra-plugin-stealth 💡 Usage: const StealthPlugin = require("puppeteer-extra-plugin-stealth") // for ESM users: // import StealthPlugin from "puppeteer-extra-plugin-stealth" puppeteer.use(StealthPlugin()) puppeteer-extra-plugin-block-resources A plugin to prevent the Puppeteer browser from loading specific resources. The supported resource types include document, stylesheet, image, media, font, script, texttrack, xhr, fetch, eventsource, websocket, manifest, other. Resource blocking can be configured both globally and locally. ⚙️ Installation: npm install puppeteer-extra-plugin-block-resources 💡 Usage: const BlockResourcesPlugin = require("puppeteer-extra-plugin-block-resources") // for ESM users: // import BlockResourcesPlugin from "puppeteer-extra-plugin-block-resources" You can then configure the resources to block globally on all pages: puppeteer.use(BlockResourcesPlugin({ blockedTypes: new Set(["image", "stylesheet"]), })) Similarly, you can locally select the resources to be blocked: puppeteer.use(BlockResourcesPlugin() const browser = await puppeteer.launch() const page = await browser.newPage() blockResourcesPlugin.blockedTypes.add("stylesheet") await page.goto("https://www.example.com/", { waitUntil: "domcontentloaded" }) puppeteer-extra-plugin-anonymize-ua A plugin to anonymize the User-Agent set by the browser controlled by Puppeteer. 🎭 It gives you the ability to strip the 'Headless' string from the Chrome user agent in headless mode and supports dynamic replacement of the user agent through a custom function. See it in action in our Puppeteer user agent guide. Discover what’s the best user agent for web scraping! ⚙️ Installation: npm install puppeteer-extra-plugin-anonymize-ua 💡 Usage: const AnonymizeUAPlugin = require("puppeteer-extra-plugin-anonymize-ua") // for ESM users: // import AnonymizeUAPlugin from "puppeteer-extra-plugin-anonymize-ua" Next, you can configure the anonymous user agent: puppeteer.use(AnonymizeUAPlugin({ stripHeadless: true, })) Also, you can set a dynamic user agent via a custom function: puppeteer.use(AnonymizeUAPlugin({ customFn: (ua) => ua.replace("Chrome", "Chromium")}) })) Puppeteer Extra Is Not a Panacea Solution Just like with Playwright, no matter how slick and customized your Puppeteer script is, advanced anti-bot systems can still sniff you out and shut you down. But how is that even possible? 🤔 The puppeteer-extra-stealth-plugin documentation breaks it down for you: Please note: I consider this a friendly competition in a rather interesting cat and mouse game. If the other team (👋) wants to detect headless chromium there are still ways to do that (at least I noticed a few, which I'll tackle in future updates). It's probably impossible to prevent all ways to detect headless chromium, but it should be possible to make it so difficult that it becomes cost-prohibitive or triggers too many false-positives to be feasible. So, while Puppeteer Extra can dodge most basic bot detection like Neo in Matrix, it can’t surely bypass Cloudflare. Sure, you could integrate a proxy into Puppeteer, but even that might not be enough. https://www.youtube.com/watch?v=7pnchjPlGTw&embedable=true The problem isn't Puppeteer itself (because let's be real, Puppeteer rocks! 🤘), but the browser it's controlling. The real solution? A powerful browser that: Operates in headed mode like a regular browser to reduce bot detection. Scales in the cloud for you, saving you time and costs in infrastructure management. Offers rotating IPs powered by one of the largest and most reliable proxy networks on the market. Automatically handles CAPTCHA solving, browser fingerprinting, cookie and header customization, and retries for optimal efficiency. Seamlessly integrates with leading browser automation libraries like Playwright, Selenium, and Puppeteer. Believe it or not, this isn't some distant dream. It's real, and it's exactly what Bright Data’s Scraping Browser has to offer! Final Thoughts Puppeteer is one of the most widely used browser automation tools in the tech world, but even superheroes have their limits. The community stepped in with puppeteer-extra, a package that gives Puppeteer some seriously cool new abilities through custom plugins. But here's the thing: while these plugins can make your scraping operation way stronger, they won’t magically turn you into a ghost 👻. Sites with advanced bot detection might still be able to block you! Bypass all anti-bots with Bright Data’s Scraping Browser—a non-detectable cloud browser that integrates seamlessly with Puppeteer. Join our mission to make the Web a public space for everyone, everywhere, even through automated scripts. Until next time, keep exploring the Internet with freedom! 🌐 As highlighted in our guide to web scraping with Puppeteer , this browser automation library is a fantastic ally for extracting data from dynamic content sites. Still, like any other tool, it has its shortcomings. That's where Puppeteer Extra steps in! web scraping with Puppeteer In this guide, we’ll introduce you to puppeteer-extra —a library that wraps puppeteer to extend it with plugin support. Get ready to take your Puppeteer scraping project to the next level! 🚀 puppeteer-extra puppeteer What’s Puppeteer Extra? Puppeteer Extra is a lightweight wrapper around puppeteer that enables plugin integration through a clean interface. Although it's not developed by the team behind Puppeteer , this community-driven project has hundreds of thousands of weekly downloads and over 6k stars on GitHub 📈. Puppeteer Extra puppeteer Puppeteer Check out the GitHub stars chart below —it’s clear that the puppeteer-extra repo has been on a steady rise in popularity over the years: GitHub stars chart below puppeteer-extra The plugins officially supported by Puppeteer Extra are: puppeteer-extra-plugin-stealth: To make it harder to detect Puppeteer as a bot. puppeteer-extra-plugin-recaptcha: To solve reCAPTCHAs and hCaptchas automatically. puppeteer-extra-plugin-adblocker: To reduce bandwidth and load times by applying a fast and efficient blocker for ads and trackers. puppeteer-extra-plugin-devtools: To make debugging the Puppeteer browser possible from anywhere. puppeteer-extra-plugin-repl: To make Puppeteer debugging and exploration easier and more enjoyable with an interactive REPL. puppeteer-extra-plugin-block-resources: To programmatically block resources like images, media files, CSS stylesheets, and more while loading pages. puppeteer-extra-plugin-flash: To allow Adobe Flash content to run on all sites without user interaction. puppeteer-extra-plugin-anonymize-ua: To anonymize the User-Agent header on all pages, with support for dynamic replacing. puppeteer-extra-plugin-user-preferences: To set custom Chrome/Chromium user preferences. puppeteer-extra-plugin-stealth : To make it harder to detect Puppeteer as a bot. puppeteer-extra-plugin-stealth puppeteer-extra-plugin-stealth puppeteer-extra-plugin-recaptcha : To solve reCAPTCHAs and hCaptchas automatically. puppeteer-extra-plugin-recaptcha puppeteer-extra-plugin-recaptcha puppeteer-extra-plugin-adblocker : To reduce bandwidth and load times by applying a fast and efficient blocker for ads and trackers. puppeteer-extra-plugin-adblocker puppeteer-extra-plugin-adblocker puppeteer-extra-plugin-devtools : To make debugging the Puppeteer browser possible from anywhere. puppeteer-extra-plugin-devtools puppeteer-extra-plugin-devtools puppeteer-extra-plugin-repl : To make Puppeteer debugging and exploration easier and more enjoyable with an interactive REPL. puppeteer-extra-plugin-repl puppeteer-extra-plugin-repl puppeteer-extra-plugin-block-resources : To programmatically block resources like images, media files, CSS stylesheets, and more while loading pages. puppeteer-extra-plugin-block-resources puppeteer-extra-plugin-block-resources puppeteer-extra-plugin-flash : To allow Adobe Flash content to run on all sites without user interaction. puppeteer-extra-plugin-flash puppeteer-extra-plugin-flash puppeteer-extra-plugin-anonymize-ua : To anonymize the User-Agent header on all pages, with support for dynamic replacing. puppeteer-extra-plugin-anonymize-ua puppeteer-extra-plugin-anonymize-ua User-Agent puppeteer-extra-plugin-user-preferences : To set custom Chrome/Chromium user preferences. puppeteer-extra-plugin-user-preferences puppeteer-extra-plugin-user-preferences On top of those, it integrates with the following community plugins: puppeteer-extra-plugin-minmax: To minimize and maximize the Puppeteer browser window in real time. puppeteer-extra-plugin-portal: To remotely view and interact with puppeteer sessions via the Chromium screencast API. puppeteer-extra-plugin-minmax : To minimize and maximize the Puppeteer browser window in real time. puppeteer-extra-plugin-minmax puppeteer-extra-plugin-minmax puppeteer-extra-plugin-portal : To remotely view and interact with puppeteer sessions via the Chromium screencast API. puppeteer-extra-plugin-portal puppeteer-extra-plugin-portal Why Do We Even Need an Extra Version of Puppeteer? No doubt, Puppeteer is one of the top headless browser libraries for scraping and testing . But let’s be honest—it has its limits, especially when facing anti-bot tech like browser fingerprinting and CAPTCHAs. Read our guide to learn how to deal with reCAPTCHA automation . browser libraries for scraping and testing reCAPTCHA automation Websites armed with anti-bot defenses can easily detect and block Puppeteer scripts. If only there was a way to extend and customize Puppeteer’s default behavior... …well, that’s exactly what Puppeteer Extra is all about! Puppeteer Extra is like a power-up for Puppeteer, adding plugin support to tackle those major drawbacks. Instead of overriding or extending everything for you, it wraps Puppeteer and lets you register only the plugins you need. 🦸 puppeteer-extra : Setup and Plugins for Web Scraping puppeteer-extra You can add Puppeteer Extra to your project's npm dependencies with: Puppeteer Extra npm install puppeteer-extra npm install puppeteer-extra ⚠️ Note : puppeteer-extra requires puppeteer to work, so make sure both packages are installed in your project. Note puppeteer-extra puppeteer Then, you have to import the puppeteer object from puppeteer-extra instead of the puppeteer library: puppeteer puppeteer-extra puppeteer const puppeteer = require("puppeteer-extra") // for ESM users: // const { puppeteer } from "puppeteer-extra" const puppeteer = require("puppeteer-extra") // for ESM users: // const { puppeteer } from "puppeteer-extra" Everything in the Puppeteer API stays the same, but you get a little extra magic ✨. The puppeteer object now exposes a use() method to plug in Puppeteer Extra plugins. puppeteer use() Time to dive into what these plugins can do, and see how they’ll level up your web scraping game! puppeteer-extra-plugin-stealth Puppeteer Extra Plugin Stealth , also known simply as Puppeteer Stealth, includes a set of configurations designed to reduce bot detection. It overrides Puppeteer’s detectable properties and settings that might expose it as a bot. Puppeteer Extra Plugin Stealth For more details, check out our guide on how to avoid getting blocked with Puppeteer Stealth . how to avoid getting blocked with Puppeteer Stealth ⚙️ Installation : Installation npm install puppeteer-extra-plugin-stealth npm install puppeteer-extra-plugin-stealth 💡 Usage : Usage const StealthPlugin = require("puppeteer-extra-plugin-stealth") // for ESM users: // import StealthPlugin from "puppeteer-extra-plugin-stealth" puppeteer.use(StealthPlugin()) const StealthPlugin = require("puppeteer-extra-plugin-stealth") // for ESM users: // import StealthPlugin from "puppeteer-extra-plugin-stealth" puppeteer.use(StealthPlugin()) puppeteer-extra-plugin-block-resources A plugin to prevent the Puppeteer browser from loading specific resources. The supported resource types include document , stylesheet , image , media , font , script , texttrack , xhr , fetch , eventsource , websocket , manifest , other . document stylesheet image media font script texttrack xhr fetch eventsource websocket manifest other Resource blocking can be configured both globally and locally. ⚙️ Installation : Installation npm install puppeteer-extra-plugin-block-resources npm install puppeteer-extra-plugin-block-resources 💡 Usage : Usage const BlockResourcesPlugin = require("puppeteer-extra-plugin-block-resources") // for ESM users: // import BlockResourcesPlugin from "puppeteer-extra-plugin-block-resources" const BlockResourcesPlugin = require("puppeteer-extra-plugin-block-resources") // for ESM users: // import BlockResourcesPlugin from "puppeteer-extra-plugin-block-resources" You can then configure the resources to block globally on all pages: puppeteer.use(BlockResourcesPlugin({ blockedTypes: new Set(["image", "stylesheet"]), })) puppeteer.use(BlockResourcesPlugin({ blockedTypes: new Set(["image", "stylesheet"]), })) Similarly, you can locally select the resources to be blocked: puppeteer.use(BlockResourcesPlugin() const browser = await puppeteer.launch() const page = await browser.newPage() blockResourcesPlugin.blockedTypes.add("stylesheet") await page.goto("https://www.example.com/", { waitUntil: "domcontentloaded" }) puppeteer.use(BlockResourcesPlugin() const browser = await puppeteer.launch() const page = await browser.newPage() blockResourcesPlugin.blockedTypes.add("stylesheet") await page.goto("https://www.example.com/", { waitUntil: "domcontentloaded" }) puppeteer-extra-plugin-anonymize-ua A plugin to anonymize the User-Agent set by the browser controlled by Puppeteer. 🎭 User-Agent It gives you the ability to strip the 'Headless' string from the Chrome user agent in headless mode and supports dynamic replacement of the user agent through a custom function. See it in action in our Puppeteer user agent guide . 'Headless' Puppeteer user agent guide Discover what’s the best user agent for web scraping! ⚙️ Installation : Installation npm install puppeteer-extra-plugin-anonymize-ua npm install puppeteer-extra-plugin-anonymize-ua 💡 Usage : Usage const AnonymizeUAPlugin = require("puppeteer-extra-plugin-anonymize-ua") // for ESM users: // import AnonymizeUAPlugin from "puppeteer-extra-plugin-anonymize-ua" const AnonymizeUAPlugin = require("puppeteer-extra-plugin-anonymize-ua") // for ESM users: // import AnonymizeUAPlugin from "puppeteer-extra-plugin-anonymize-ua" Next, you can configure the anonymous user agent: puppeteer.use(AnonymizeUAPlugin({ stripHeadless: true, })) puppeteer.use(AnonymizeUAPlugin({ stripHeadless: true, })) Also, you can set a dynamic user agent via a custom function: puppeteer.use(AnonymizeUAPlugin({ customFn: (ua) => ua.replace("Chrome", "Chromium")}) })) puppeteer.use(AnonymizeUAPlugin({ customFn: (ua) => ua.replace("Chrome", "Chromium")}) })) Puppeteer Extra Is Not a Panacea Solution Just like with Playwright , no matter how slick and customized your Puppeteer script is, advanced anti-bot systems can still sniff you out and shut you down. But how is that even possible? 🤔 Just like with Playwright The puppeteer-extra-stealth-plugin documentation breaks it down for you: puppeteer-extra-stealth-plugin documentation Please note: I consider this a friendly competition in a rather interesting cat and mouse game. If the other team (👋) wants to detect headless chromium there are still ways to do that (at least I noticed a few, which I'll tackle in future updates). It's probably impossible to prevent all ways to detect headless chromium, but it should be possible to make it so difficult that it becomes cost-prohibitive or triggers too many false-positives to be feasible. Please note: I consider this a friendly competition in a rather interesting cat and mouse game. If the other team (👋) wants to detect headless chromium there are still ways to do that (at least I noticed a few, which I'll tackle in future updates). It's probably impossible to prevent all ways to detect headless chromium, but it should be possible to make it so difficult that it becomes cost-prohibitive or triggers too many false-positives to be feasible. So, while Puppeteer Extra can dodge most basic bot detection like Neo in Matrix, it can’t surely bypass Cloudflare . Sure, you could integrate a proxy into Puppeteer , but even that might not be enough. bypass Cloudflare integrate a proxy into Puppeteer https://www.youtube.com/watch?v=7pnchjPlGTw&embedable=true https://www.youtube.com/watch?v=7pnchjPlGTw&embedable=true The problem isn't Puppeteer itself (because let's be real, Puppeteer rocks! 🤘), but the browser it's controlling. The real solution? A powerful browser that: Operates in headed mode like a regular browser to reduce bot detection. Scales in the cloud for you, saving you time and costs in infrastructure management. Offers rotating IPs powered by one of the largest and most reliable proxy networks on the market. Automatically handles CAPTCHA solving, browser fingerprinting, cookie and header customization, and retries for optimal efficiency. Seamlessly integrates with leading browser automation libraries like Playwright, Selenium, and Puppeteer. Operates in headed mode like a regular browser to reduce bot detection. Scales in the cloud for you, saving you time and costs in infrastructure management. Offers rotating IPs powered by one of the largest and most reliable proxy networks on the market. Automatically handles CAPTCHA solving, browser fingerprinting, cookie and header customization, and retries for optimal efficiency. Seamlessly integrates with leading browser automation libraries like Playwright, Selenium, and Puppeteer. Believe it or not, this isn't some distant dream. It's real, and it's exactly what Bright Data’s Scraping Browser has to offer! Scraping Browser Final Thoughts Puppeteer is one of the most widely used browser automation tools in the tech world, but even superheroes have their limits. The community stepped in with puppeteer-extra , a package that gives Puppeteer some seriously cool new abilities through custom plugins. puppeteer-extra But here's the thing: while these plugins can make your scraping operation way stronger, they won’t magically turn you into a ghost 👻. Sites with advanced bot detection might still be able to block you! Bypass all anti-bots with Bright Data’s Scraping Browser—a non-detectable cloud browser that integrates seamlessly with Puppeteer . Join our mission to make the Web a public space for everyone, everywhere, even through automated scripts. integrates seamlessly with Puppeteer Until next time, keep exploring the Internet with freedom! 🌐