Bundlejs: An Online Esbuild-Based Bundler

Written by okikio | Published 2022/05/23
Tech Story Tags: javascript | typescript | bundler | pwa | webdev | compression | esbuild | devtool

TLDRbundlejs is a quick and easy way to treeshake, bundle, minify, and compress (in either gzip or brotli) projects, while receiving the total bundles' file size. It's easier to debug errors, easier to configure your bundles, and you can view a visual analysis of bundles. You can bundle offline (so long as the module has been used before) The benefits of using bundlejs are: It's been easier to diagnose errors and verify the resulting bundled code. There will be a follow-up article to this one, going into the technical nitty gritty on how the project works.via the TL;DR App

Introduction

bundlejs (pronounced bundle js) is a quick and easy way to tree shake, bundle, minify, and compress (in either gzip or brotli) your typescript, javascript, jsx, and npm projects, while receiving the total bundles' file size.

bundlejs aims to generate more accurate bundle size estimates by following the same approach that bundlers use:

  • Doing all bundling locally
  • Outputting the tree shaken bundled code
  • Getting the resulting bundle size

The benefits of using bundlejs are:

  1. It's easier to debug errors

  2. You can verify the resulting bundled code

  3. The ability to configure your bundles

  4. The ability to tree shake bundles

  5. The ability to view a visual analysis of bundles

  6. You can bundle offline (so long as the module has been used before)

  7. Supports different types of modules from varying Content Delivery Networks (CDNs), e.g., CDNs ranging from deno modules to npm modules to random GitHub scripts, etc...

This blog post is meant to highlight some of the most important changes, as well as to give some insight into how bundlejs works in the background, and to act as the docs for bundlejs.

📒Note: There will be a follow-up article to this one, going into the technical nitty gritty on how bundlejs works, and how you can use what I've learned from this project to either create your own online bundler or an es build-wasm backed js repl.

😅 TL;DR: this blog post is rather long, so take a look at the bundlejs.com website first, then skim through this blog post and make sure to check out the images and the code examples, those can help cut down on confusion and reduce the required reading time.

Quick feature run down

https://www.youtube.com/watch?v=5FTricK1peg

This video runs through all the major features of bundlejs (there is audio but I don't have a good mic 😅)

Bundling, Tree shaking, and Minification

  • Bundling is the process of efficiently concatenating modules together into one file which we call a bundle.

  • Tree shaking is the process of a bundler traversing the modules to be bundled and removing unused code.

  • Minification is the process of shrinking the amount of code necessary to have a functional program, e.g. removing blank space or reducing variable names, etc...

bundlejs uses esbuild and its incredible ability to bundle, transform, transpile, minify, tree shake, and traverse files. Specifically, bundlejs uses esbuild-wasm, which is able to access a subset of those features, with the key limitations being:

  1. Npm only runs on node, so no package.json or npm install (a-la, a joke about using StackBlitz WebContainers to run node on the browser)

  2. Browsers don't work the way nodejs does. They don't have an easy way to access the file system, so storing and accessing files isn't practical. The way esbuild would normally work when installed on node just causes issues on the web

  3. Due to the limitation of esbuild-wasm when running on a browser (no npm, and node nodejs), the only option is for modules to come from the web, but esbuild doesn't natively support importing http(s)://... modules, so a different solution is required.

To solve each of these problems, esbuild's plugin system comes in clutch. I created a total of 4 plugins to solve these limitations, they are:

  1. HTTP plugin - Fetches and caches modules

  2. CDN plugin - Redirects npm package imports (sometimes referred to as bare imports) to Content Delivery Network (CDN) URLs for fetching

  3. EXTERNALS plugin - Marks certain imports/exports as modules to exclude from bundling

  4. ALIAS plugin - Aliases certain imports/exports to modules of a different name.

Content Delivery Networks (CDNs) are a great way to distribute code all over the world at fast speeds. In the context of bundlejs, CDNs represent online repositories of code that bundlejs can fetch from.

For example, unpkg.com is a fast global content delivery network for everything on npm. It's used to quickly and easily load any file from any package on npm using a URL like: https://unpkg.com/package-name@version/file.js, a similar thing would apply for skypack.dev, esm.sh, etc...

In a later blog post, I will delve deeper into the technical details of how these plugins work, but for now, just keep in mind that these plugins assist esbuild-wasm to create javascript bundles.

ℹ️ Info: This is the impact tree shaking and minifying a bundle has on bundle size:

treeshaken

vs.

non-treeshaken

Console

In previous versions of bundlejs.com, I encouraged devs to use the devtools console for viewing console logs, and for a while, I thought it was an ok experience, but I started realizing that it was inconvenient and not very mobile-friendly.

Initially, I thought creating a virtual console would be a large undertaking, so I delayed adding a custom console for quite some time. Well, in March of this year, inspired by @hyrious's esbuild-repl, I finally did it 👏.

Console Results

The console functions by listing out details of the build pertinent to users, e.g., fetched packages, errors, etc...This includes bundle result info.

Fetching Packages

By default, the console will display the progress for fetching packages It's often the best way to diagnose errors and issues, as well as to find points of improvement.

For some packages (ahem @babel/core), there are too many sub-packages. Having the virtual console handle that many logs will eat up too much memory, and/or slow down less powerful devices; so I limit the number of logs to 250 and when that limit is passed, bundlejs will show this friendly message:

📒 Note: you can still access the full console log from the devtools console, even if the virtual console does any kind of truncating.

You can change the maximum amount of logs allowed in the config via:

{
    "esbuild": {
        "logLimit": 500
    }
}

Console Errors & Warnings

Errors look like this:

Warnings look like this:

Console Buttons

The consoles were also given buttons to make them easier to navigate, they are these right here

This is the scroll to top button. As new console logs are introduced, the console automatically sticks to the bottom. Some users may find this behavior annoying, so this button offers a quick and easy opt-out by taking the user straight to the top of the console.

This is the scroll to bottom button. Basically, a get to the bottom as quickly as possible button, it's meant for quickly checking out the final bundle result.

This is the expand/collapse button, it expands/collapses all errors and warnings quickly, making it easier to navigate through a large set of errors.

This is the clear console button, it clears the console of contents leaving the console looking like this:

Console Extras

Sticky Console: Sticks console scroll position to the bottom for new logs. If you scroll ~50px away from the bottom, this behavior no longer applies; if you scroll back to the bottom, the behavior will apply again.

Pet-peeve alignment: I get so annoyed when things that can align don't align, so I built into the console to align by default with the bundlejs result section on a large enough screen, e.g. laptops, tablets, desktops, etc...

Clickable console links: Exactly as it sounds, clickable console links highlight URLs that start with http(s)://..., they function just like how vscode and the devtools do and act as an easy way to access console URLs without having to copy and paste the URL manually.

It does have some limitations, namely, it sometimes has difficulty recognizing which characters are links and which are not, e.g.,

I couldn't find an example while making this blog post 🤣, when/if (hoping it's an if rather than a when) I find one, I'll update this post.

Log bundle results: The bundle results, e.g., the time it takes to create the bundle, the initial bundle size, the compressed bundle size, and more are logged to the console.

Input and Output Tabs

With the addition of the console, I wanted to ensure that the editors weren't unwieldy, so I created a tab bar for the input, output, and config editor. The tab bar allows for quick access to all editors while ensuring that the console is always available.

Configuration

In v0.2, I added support for custom configurations (configs), it supports most of esbuild's build options, as well as some added options to change the default CDN and the compression algorithm.

The default config is:

{
  "cdn": "https://unpkg.com",
  "compression": "gzip",
  "esbuild": {
      "target": [ "esnext"],
      "format": "esm",
      "bundle": true,
      "minify": true,
      "treeShaking": true,
      "platform": "browser"
  }
}

When you click the share button, it will also share the custom config you've setup, e.g.,

{
  "cdn": "https://unpkg.com",
  "compression": "lz4",
  "esbuild": {
      "target": [ "es2018" ],
      ...
  }
}

The config above would result in this share URL bundlejs.com/?q=@okikio/animate&config={"compression":"lz4","esbuild":{"target":["es2018"]}}.

Notice how cdn is missing from the share URL, that's because bundlejs smartly decides on which config to send as a part of the share URL based on how different the new config is from the default config.

📒 Note: There are 3 available compression algorithms, brotli, gzip, and lz4.

CDN Hosts

Content Delivery Networks (CDNs) are a great way to distribute code all over the world at fast speeds. In the context of bundlejs, CDNs represent online repositories of code that bundlejs can fetch from.

For example, unpkg.com is a fast, global content delivery network for everything on npm. It's used to quickly and easily load any file from any package on npm using a URL like: https://unpkg.com/package-name@version/file.js, a similar thing would apply for skypack.dev, esm.sh, etc...

By default, bundlejs lets you enter code like this:

export * from "@okikio/animate";

But behind the scenes, bundlejs auto fetches that specific package from a CDN namely, unpkg.

In older versions of bundlejs, the default CDN used to be skypack, but because skypack doesn't have easy access to the package.json of node packages, I switched to using unpkg as the default CDN.

With later updates, bundlejs received the ability to update the default cdn on a global or local scale.

Technical details and more info...

You can choose CDNs by:

  1. (Global CDN) Setting the CDN config to a different CDN host, e.g.

    {
    "cdn": "https://cdn.esm.sh",
    // OR
    "cdn": "skypack"
}

  2. (Local CDN) Using the CDN host as an inline URL scheme, e.g.

    export { animate } from "skypack:@okikio/animate"; 
//                       ^^^^^^^  https://cdn.skypack.dev/@okikio/animate

  3. There are a total of 8 supported inline CDN host url schemes:

After determining the CDN to use, the next step is to determine if the CDN host supports npm style modules, examples of which are unpkg, skypack, esm.sh, etc...

The factors involved in determining that a CDN host supports npm style modules are that the CDN host supports:

  1. The CDN supports package versioning through the @version URL tag (e.g. react@18).

  2. The CDN can load node packages, package.json file.

📒 Note: Without the package.json, you can't load subpath imports, plus it becomes more difficult to determine the correct exported modules to bundle with.

⚠️ Warning: If the chosen CDN doesn't support the package.json file, and it isn't a npm style CDN host, then bundlejs will switch to trying to guess package versions; this may lead to inaccurate bundles with the wrong versions of packages.

In a later blog post, I will delve deeper into the technical details of how the esbuild CDN plugin works to determine which CDN host to use, but for now, you just keep in mind that the CDN plugins assist esbuild-wasm in resolving CDN host URLs.

Compression Algorithms

bundlejs offers the options of bundling using:

  1. brotli - results in the smallest bundle size, but it's the slowest

  2. gzip - results in the 2nd smallest bundle size, but it's faster than brotli (default)

  3. lz4 - results in the largest bundle size, but it's the fastest bundle algorithm.

📒 Note: Each compression algorithm has it's own story.

The Brotli Problem

brotli is a compression algorithm that compresses data really well, however, it's very slow compared to other alternatives. Adding brotli was quite an undertaking with lots of ups and downs, but thanks to Lewis Liu on Twitter, I was able to use deno-brotli to include a WASM version of brotli in bundlejs.

Learn the story behind brotli support...

No shade to the original creators of brotli-wasm and wasm-brotli (different packages, similar name), but the way both packages handle WASM forces devs to use webpack (which wasn't happening, I value my time tooooo much for that). After multiple attempts and 6 months of work later, I finally found deno-brotli.

An example of the way WASM is used in both packages (brotli-wasm and wasm-brotli) is as follows:

import WASM from "./bg.wasm";
// ...

In all honesty, it's not fair for me to blame the creators of brotli-wasm and wasm-brotli, it's not their fault. The fault lies in the js ecosystem not yet finding an interoperable solution for working with WASM. That's one of the key reasons I'm very thankful for Lewis Liu pointing out deno-brotli.

deno-brotli does 2 things right, they are:

  1. It compresses the huge WASM file required for deno-brotli into an lz4 compressed string, which can then be decompressed by lz4 allowing for easy storage of the WASM as a js file (the result is great build tools support as the WASM is just a string inside a JS file, plus it solves the ecosystem problem really well).

    For lz4 support, bundlejs is using deno-lz4, which also runs via WASM, but the way deno-lz4 compress itself is slightly different than deno-brotli. I'd highly encourage you to take a look at the source code for deno-lz4, it's really informative.

  2. By having the WASM as a js file, you can actually preload the WASM as a js module 🤯

You can read through this tweet thread to learn more:

https://twitter.com/okikio_dev/status/1498898909879422977?s=20&t=iUL2EzC810kgGM6tn8nJeg

Default to Gzip

bundlejs has used gzip as the default for quite some time. bundlejs used to use pako, but thanks to a discussion with @matthewcp who rightfully pointed out the (De)compression Stream API, I started looking into alternatives to pako.

https://twitter.com/matthewcp/status/1503704061853450242?s=20&t=CDGJmTts_m2A9H7cyZfmew

Learn the story behind gzip support...

Thanks to the conversation with @matthewcp, I actually did some further research into pako alternatives> I have 3 possible alternatives, they are denoflate (which uses WASM), deno-compress (which uses js), and CompressionStream (which is built into browsers), yay fun 🎉😅.

I eventually chose to replace pako with denoflate as the default compression algorithm for gzip; it's a bit faster and smaller than pako.

LZ4, Gotta Go Fast

In order to use WASM in a portable way, deno-brotli would compress the WASM binary file into a base64 string using lz4 as the compression algorithm. I saw the opportunity to add another compression algorithm, so I used the lz4 (deno-lz4) implementation used to compress the brotli WASM binary string (see #the-brotli-problem for more info.), it was by far the easiest compression algorithm to add to bundlejs 🤣.

Compression Quality

You can set the quality of the compression (from a scale of 1 to 11, with 1 being the least compressed and 11 being the most compressed), you can set the compression quality for any of the compression algorithms mentioned above by setting the compression config option to:

{
    "compression": {
        "type": "brotli",
        "quality": 11
    }
}

You can check out a demo here, bundlejs.com/?config={"compression":{"type":"brotli","quality":11}}.

Aliases and Externals

Aliases are a way to redirect certain packages to other packages, e.g. redirecting the fs to memfs, because fs isn't supported on the web, etc... This wasn't a direct feature request, but I felt it would be a good addition.

Externals are a direct feature request issue#13, it took a while but a good solution is finally a part of bundlejs, you use it the way you'd use the esbuild externals config option.

More details...

You use aliases like this:

{
    "alias": {
        "@okikio/animate": "@babel/core"
    }
}

You can try it out below, bundlejs.com/?config={"alias":{"@okikio/animate":"@babel/core"}}.

You use externals like this:

{
    "esbuild": {
        "external": ["@okikio/animate"]
    }
}

You can try it out below, bundlejs.com/?config={"esbuild":{"external":["@okikio/animate"]}}.

Check out a complex example of using the external config bundlejs.com/?q=@babel/core&config={"esbuild":{"external":[...]}}

No one else can understand my pain...I'm adding more feature to bundlejs as I'm writing this blog post, so it's just getting longer and longer and longer, etc.... 😅

Esbuild Config Options

esbuild config options are exactly how they sound, however, with esbuild running on the browser, there are some limitations on what esbuild can do. Due to the lack of native filesystem access, some options don't work or are rendered obsolete.

The supported esbuild build options are:

Simple options

Advanced options

Quite a bit to work with, I'd say.

Editor Buttons + Extra Features...

The editor buttons add extra functionality to the editor; they enable easy access to common editor tasks.

The current list of editor buttons are:

Editor panel toggle

Toggles on or off the editor buttons, leaving more space for the code editor. It looks like this when the editor buttons are hidden:

Clear editor button

Clears the editor of all its contents.

Format code button

Cleans up any messy code it finds. It uses dprint to format the input and output editor code but falls back to monaco-editors baked in formatter for the config editor.

Reset code button

Resets the editor to its initial state.

  • For the input editor, it resets it to this:

    // Click Build for the bundled, minified and compressed package size
export * from "@okikio/animate";

  • For the output editor, it resets it to this:

    // Output

  • For the config editor, it resets it to this:

    {
    "cdn": "https://unpkg.com",
    "compression": "gzip",
    "analysis": false,
    "esbuild": {
        "target": [
            "esnext"
        ],
        "format": "esm",
        "bundle": true,
        "minify": true,
        "treeShaking": true,
        "platform": "browser"
    }
}

Copy code button

Copies the editor’s code, it's exactly as it sounds (what were you expecting? 🤣). When you copy code from the editor using the copy button, this delightful little message appears:

Wrap code button

Toggles between wrapped and unwrapped code. Wrapping is all about making the editor’s code wrap around the constraints of the editor’s bounding box, removing the need to scroll horizontally to view all the code.

This is how wrapped code looks:

This is how unwrapped code looks:

Bonus features

Bonus: You can access monaco's built in command palette by pressing F1, e.g.

Extra Bonus: You can use many of the code shortcuts vscode has by just right clicking while in focus on the monaco-editor, e.g.

Drag Handles...Interactive Fun

https://www.youtube.com/watch?v=AMt01tFstik&feature=youtu.be

The new drag handles enable a more interactive experience with the editor; they are a little like the drag handles in vscode, but mobile-friendly.

bundlejs being mobile friendly isn't a huge focus point for the project, but it's nice to have if you ever find yourself in the need for bundlejs while on a mobile or touch enabled device.

JSX Support

JSX is now officially supported in bundlejs 🎉.

Ignore the red error lines, for some reason the monaco code editor doesn't want to work well with JSX 😅

To use JSX, you need to set the jsxFactory and the jsxFragment config options according to the JSX-based framework you are using.

e.g. for Preact, the following config would be used:

{
    "esbuild": {
        "jsxFactory": "h",
        "jsxFragment": "Fragment"
    }
}

Try out the preact demo

Sharing Bundle Sessions

To share bundle sessions* between multiple users (while avoiding the need for a server) we need a static and local way to store and share code between users. To solve this problem, I decided to encode the bundle session* information right into the URL, this was because I wanted the entire project to run offline, and I didn't want the high maintenance cost of a server and database.

*sessions are the specific state of bundlejs at a specific time, they are not the entire bundle session history, just the input code and the bundle configuration at the time the share button is clicked.

Technical details...

A high-level summary of how this works is that users make a change in the input code editor, that change then gets saved and encoded into the URL. The URL can then be used to create replays of the bundle session.

A sample session URL is:

/?q=(import)@okikio/emitter,(import)@okikio/animate,(import)@okikio/animate,(import)@okikio/animate,(import)@okikio/animate,@okikio/animate,@okikio/animate,@okikio/animate,@okikio/animate&treeshake=[T],[{ animate }],[{ animate as B }],[ as TR],[{ type animate }],[],[{ animate as A }],[ as PR],[{ animate }]&text="export  as PR18 from \"@okikio/animate\";\nexport { animate as animate2 } from \"@okikio/animate\";"&share=MYewdgziA2CmB00QHMAUAiAwiG6CUQA&config={"cdn":"skypack","compression":"brotli","esbuild":{"format":"cjs","minify":false,"treeShaking":false}}&bundle

The resulting input code of this bundle session url is this:

// Click Build for the Bundled, Minified & Compressed package size
import T from "@okikio/emitter";
import { animate } from "@okikio/animate";
import { animate as B } from "@okikio/animate";
import  as TR from "@okikio/animate";
import { type animate } from "@okikio/animate";
export  from "@okikio/animate";
export { animate as A } from "@okikio/animate";
export  as PR from "@okikio/animate";
export { animate } from "@okikio/animate";
console.log("Cool")
export  as PR18 from "@okikio/animate";
export { animate as animate2 } from "@okikio/animate";

with a config of:

{
    "cdn": "skypack",
    "compression": "brotli",
    "esbuild": {
        "target": ["esnext"],
        "format": "cjs",
        "bundle": true,
        "minify": false,
        "treeShaking": false,
        "platform": "browser"
    }
}

The URL breakdown is:

/?
q=(import)@okikio/emitter,(import)@okikio/animate,(import)@okikio/animate,(import)@okikio/animate,(import)@okikio/animate,@okikio/animate,@okikio/animate,@okikio/animate,@okikio/animate&
treeshake=[T],[{ animate }],[{ animate as B }],[* as TR],[{ type animate }],[*],[{ animate as A }],[* as PR],[{ animate }]&
text="export * as PR18 from \"@okikio/animate\";\nexport { animate as animate2 } from \"@okikio/animate\";"&
share=MYewdgziA2CmB00QHMAUAiAwiG6CUQA&
config={"cdn":"skypack","compression":"brotli","esbuild":{"format":"cjs","minify":false,"treeShaking":false}}&
bundle

  • q or query represents the module, e.g. react, vue, etc...

    You can add (import) in-front of a specific module to make it an import instead of an export

  • treeshake represents the export/imports to treeshake.

    The tree shake syntax allows for specifying multiple exports per package, through this syntax:

    "[{ x,y,z }],[*],[* as X],[{ type xyz }]" 
// to
export { x, y, z } from "...";
export * from "...";
export * as X from "...";
export { type xyz } from "...";

    The square brackets represent separate packages, and everything inside the square brackets, are the exported methods, types, etc...

  • text represents the input code as a string (it's used for short input code)

  • share represents compressed string version of the input code (it's used for large input code)

  • config represents the bundle configuration to use when building the bundle

  • bundle tells bundlejs to bundle the input code on start-up. This isn't on by default for security reasons. I want to discourage people from sending large complex bundles that crash browsers or that take a long time to load, especially before the input code is properly verified as non-malicious. So, if you want to bundle the code on startup, you have to manually add &bundle to the end of the URL yourself.

The reason why I decided on this syntax is because it allows for a lot of flexibility and transparency concerning what is being bundled. I also wanted to make it easy to share bundle sessions between users.

Bundle Analysis

bundlejs can analyze and visually represent bundles as easy to navigate and easy-to-understand charts.

Using a port of esbuild-visualizer and rollup-plugin-visualizer by @bardadymchik, I added the ability to visualize bundles, this feature comes from a feature request by @atomiks on issue#22, the issue is still open you can make suggestions to improve this feature.

The bundle analysis charts are displayed right under the editor, like so:

The charts displayed comes in 3 distinct flavors:

Treemap Chart

Treemap charts are the most memorable form of bundle analysis chart; the inspiration behind this chart is webpack-bundle-analyzer. webpack-bundle-analyzer is the progenitor of bundle analyzers, and a great inspiration to the approach bundlejs took to creating charts.

Treemap charts:

  1. Help you realize what's really inside your bundle

  2. Find out what modules make up the most of its size

  3. Find modules that got there by mistake

  4. Optimize it!

Source: github.com/webpack-contrib/webpack-bundle-analyzer

Source: github.com/webpack-contrib/webpack-bundle-analyzer

Though the bundlejs treemap chart is less powerful than the webpack-bundle-analyzer's treemap chart, it is simpler, and faster to use (bundlejs uses esbuild and the bundle analysis is easily available online).

Network Chart

Network charts don't change a lot from the treemap chart, however, they do offer a unique perspective on the impact of the relative sizes of modules in a bundle.

Sunburst Chart

Similar to network charts, sunburst charts don't reinvent the wheel. They accomplish similar things to the treemap chart, though the difference comes in how they represent the relative size of modules in bundles.

Sunburst charts use pie charts to represent bundle sizes, it aids in understanding just how much of the total bundle size certain modules take up.

Technical details...

📒 Note: All analysis charts support the gzipped and brotli compressed sizes of bundles! When analyzing a bundle it will choose either gzip or brotli based on the compression type.

e.g. A config of:

{ 
  "compression": "gzip" 
}

will use gzip compression for the charts, resulting in:

Analytics

When I initially built the project, I only used a simple page view counter. I wanted to view how popular the project was without violating user privacy, it worked but I felt it could be better. So, I decided to also use umami as a privacy-preserving, cookieless, open-source, Google Analytics alternative, to which the analytics are public for anyone to view.

Extra detail...

For bundlejs, a self-hosted version of umami is used, this is to ensure user data is kept private and secure. When trying to setup the self-hosted version of umami, I found that the article Setting up Umami with Vercel and Supabase by Jakob Bouchard, was a great help.

The analytics are publicly available, check them out at, analytics.bundlejs.com/share/bPZELB4V/bundle

Or click the page visit counter:

📒Note: bundlejs is still using a page view counter, the view counter is powered by countapi (to the best of my knowledge, countapi is now deprecated, however, the servers for the project are still up and running. So, I'll keep using the project until I switch to using umami for page views as well as general analytics).

https://dev.to/jakobbouchard/setting-up-umami-with-vercel-and-supabase-3a73

Discussions and Support

To encourage discussion, give support, and gain feedback, I added a comment section to bundlejs, I used giscus for this.

Initialy, when I created the bundlejs project, I also created a GitHub Discussion for it as well. I didn't want to have the overhead of having to manage a Discord server, so I choose GitHub Discussions for chats about bundlejs.

The problem is that no one really uses GitHub Discussions, so I integrated it right into the website itself via giscus. This was so new users can easily interact with others, get support from me, and leave me feedback.

Technical details...

giscus is an open-source comments system powered by GitHub Discussions; it lets visitors leave comments and reactions on your website via GitHub! It was heavily inspired by utterances.

For bundlejs, I'm using a self-hosted version of giscus, this was for security reasons mostly. When trying to set up giscus for bundlejs, the self-hosting docs on the GitHub repo are very helpful. I highly suggest anyone thinking of using giscus to read it, it gives insight into how giscus works on the backend.

You can check out the integrated giscus comments under the link, bundlejs.com/#discus.

As of right now, the comments section is looking really bare and basic, why not leave your mark. Leave a comment with what you love and what you think needs improvement in bundlejs, I'll go through them and try to integrate your ideas into bundlejs.

Security and Performance

Security and performance are critical quality areas for bundlejs. In order to bundle modules together, bundlejs has to fetch multiple sets of modules from all over the internet, while ensuring that malicious actors don't get involved, and that esbuild-wasm isn't taken advantage of to crash other devices.

Some really...really large modules can take up to 4+ GB of memory to be bundled properly by esbuild-wasm.

The security criteria I set for bundlejs were:

  1. Don't leak personal user info.
  2. Don't go through a central server, e.g., the ability to get node modules from various sources.
  3. Ensure people always know what packages they are bundling.
  4. Ensure people can't use bundlejs to maliciously slow down browsers.

To ensure I met the security criteria set:

  1. I use strict Content Security Policies (CSP) for bundlejs, ensuring no unintended 3rd party can get involved.

  2. I self-host as many of the 3rd party scripts as I can. By enclosing the number of hands involved in bundlejs.com, I reduce the chance that personal information is leaked.

  3. I use sandboxing techniques, e.g., Web Workers and Shared Workers to ensure the main thread runs at a smooth 60fps while avoiding access to the DOM.

    Web Workers and Shared Workers are scripts that run on a seperate thread. By using Workers, I am able to isolate potentially malicious code while ensuring that the main-thread isn't affected.

    Most of the uses of Workers were Shared Workers. Shared Workers reduce the performance impact of multiple instances of the bundlejs site/web app running on the same device.

  4. To reduce the chance of bundlejs being used to maliciously slow down browsers, I ensure the share URL is easy to read and understand, and by default disable auto-bundling for shared URLs.

The current browser landscape for Shared Worker support is spotty at best.

The support table looks like this:

Browser

Shared Workers

Module Workers

Firefox

Yes

No

Chrome

Yes

Yes

Safari

Yes*

Yes

* Support for Safari is currently experimental but should be coming in later versions

Module Workers are esmodules that run in Workers.

📒 Note: I built a Shared Worker polyfill @okikio/sharedworker. It's a small, but simple polyfill that falls back to a regular Web Worker if Shared Workers are not supported.

You can use it while waiting for the next version of Safari to support Shared Workers, or while supporting older versions of Safari.

⚠️ Warning: The Shared Worker polyfill doesn't handle module workers, you will still need to somehow compile your modules to non-esm versions to support workers in Firefox.

You can view how bundlejs handles module workers in the bundlejs source code, you may also wish to view the astro-repl source code to see how it handles module workers.

Most of the other security policies are passive in nature, e.g.,

  • bundlejs only bundling on page load if the URL has ?bundle in it.
  • bundlejs enforcing https:// for all requests, including for iframes, etc...
  • Only have properly vetted CDN hosts for bundlejs by default.
  • etc...

Tips and Tricks

Top tier tip, follow me (@okikio_dev) and bundlejs (@jsbundle) on twitter; shameless plug 🤣.

I do post announcements and updates on these accounts, as well as small tips and tricks that help in making the most use of bundlejs.

  • When bundling packages that also export CSS and other external files, bundlejs.com now checks the gzip/brotli size of these external files, however, it won't output the external files' code. This behavior may change in the future, but for now, that is the approach I am going with. Keep in mind that this isn't a bug, however, if it causes confusion, I am willing to change this behavior.

  • Tree shaking is available, but not all CDNs support access to each packages package.json, so there might be slight package version conflicts. The only verified CDN with access to the package.json is https://unpkg.com. The other CDNs that are used either pre-bundle the code for us (this is hit or miss depending on the package) or they aren't full npm CDNs e.g. https://deno.land or https://raw.githubusercontent.com.

  • Check the full devtools console for error messages and warnings if you are having trouble debugging an issue in bundlejs, or even better yet, enable esbuilds verbose logging when trying to debug issues, e.g.

    {
    "esbuild": {
        "logLevel": "verbose"
    }
}

    Check out a demo.

  • You can use custom protocols to specify which CDN's specific imports/exports module you should use. If an error occurs, such that you can't bundle a package properly, I highly suggest switching CDNs via either custom protocols or by changing the cdn config option. I recommend using custom protocols instead of the cdn config option when trying to debug issues with a CDN:

  • e.g.

    {
    "cdn": "unpkg"
}

  • or

    export * from "unpkg:typescript";

    Try using custom protocols to solve this example issue on bundlejs.

  • For some packages, a soft error occurs where the default export is excluded from the tree shaken bundle. The solution for this is to manually include the default export like so”

    export * from "skypack:solid-dismiss";
// and
export { default } from "skypack:solid-dismiss";

If you have a tip and trick you would like to share, post a comment below, or send me a tweet!

Contribute

The codebase is currently quite disorganized, so I'd suggest direct messaging me on Twitter or starting a GitHub Discussion to discuss ways to contribute.

There is a lot of stuff happening on the bundlejs project, and it can be very overwhelming. If you think you can still contribute, by all means, please do! I will eventually get to writing detailed docs on how to contribute, and how everything works in the backend; look forward to it.

You can use a pre-made Gitpod dev environment to quickly get started with the project or to contribute quick changes to the project.


If you love the project, I'd welcome it if you'd spread the word, my goal is to make bundlejs a viable alternative/replacement for bundlephobia and even local bundlers. But right now, the project is so small that most people who'd benefit from it don't know about it. I'd love to see people using it.

Last note, bundlejs is now on OpenCollective, so if you'd like to contribute to it financially, it'd be appreciated.

Conclusion

bundlejs, a quick and easy way to tree shake, bundle, minify, and compress (in either gzip or brotli) your typescript, javascript, jsx, and npm projects, while receiving the total bundles' file size.

bundlejs aims to generate more accurate bundle size estimates by following the same approach that bundlers use:

  • Doing all bundling locally
  • Outputting the tree shaken bundled code
  • Getting the resulting bundle size

The benefits of using bundlejs are:

  1. It's easier to debug errors

  2. You can verify the resulting bundled code

  3. The ability to configure your bundles

  4. The ability to tree shake bundles

  5. The ability to view a visual analysis of bundles

  6. You can bundle offline (so long as the module has been used before)

  7. Supports different types of modules from varying Content Delivery Networks (CDNs), e.g., CDNs ranging from deno modules to npm modules to random GitHub scripts, etc...

The next time you need to bundle a project or you need to know the bundle size of a project, give bundlejs.com a try.

📒Note: There will be a follow up article to this one, going into the technical nitty gritty on how bundlejs works and how you can use what I've learned from this project to either create your own online bundler or an esbuild-wasm backed js repl.

Photo by Okiki Ojo, you can find the image on Dropbox.

Originally published on blog.okikio.dev

Also published on Hackernoon and dev.to

Written by okikio | Software Developer @vercel, Maintainer @astro.build, Creator of bundlejs.com & inthistweet.app
Published by HackerNoon on 2022/05/23
ABOUTHow hackers start their afternoons. We publish tech stories and build publishing software.

READEntirely free library read by hundreds of millions to learn anything about any technology.

WRITEHackerNoon elevates quality technology writing far and wide across the interwebs.

PARTNERHackerNoon works directly with tech companies to place ads by content relevancy