Large Files Transfers Between Parts of Chrome Extensions for Manifest V3

As you know, I am an author of the PerfectPixel browser extension. Recently, I transferred it from Manifest V2 to Manifest V3, and during the process, re-architected the messaging system due to the changes made in the new messaging API.

The challenge is that the messaging protocol has a message size limit now, so we have to deal with that limitation.

In this article, I'll outline Manifest V3 messaging fundamentals, overhaul the problem, and then share the solution I've made. The solution is published as an NPM package; the source code is available on GitHub. You can find the link at the end of the article.

Chrome Extensions Messaging Fundamentals

Chrome extension compliant with Manifest V3 consists of several parts: background service worker (replacement for background pages in Manifest V2), content scripts, and different web pages - popup, settings, offscreen. All of those pieces have a universal API for communication with each other: chrome.runtime.messaging. It uses a pub-sub model.

chrome.runtime.sendMessage(message: any, callback: function) - method broadcast message to all parts of your extension. The second argument is used to handle the response.

chrome.runtime.onMessage.addListener((message: any, sender: MessageSender, sendResponse: function) => boolean) method registers a listener. The third argument of the listener function is used to send a response to the sender, sync, or async. The return value of the listener is used to determine if the response is expected to be synced or asynced.

Example

content script:

chrome.runtime.sendMessage({
    type: 'foo-bar'
    content: 'foo'
}, (response) => {
    // response === 'bar'
    ...
});

service worker, sync listener:

chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
    switch(message.type) {
        case 'foo-bar':
            sendResponse('bar');
            break;
        default:
            break;
    }
    return false; // sync
})

service worker, async listener:

chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
    switch(message.type) {
        case 'foo-bar':
            somePromiseToExecute().then(() => {
                sendResponse('bar');
            })
            break;
        default:
            break;
    }
    return true; // async
})

The Problem

The message sent via sendMessage has a maximum length that cannot be exceeded or the message won't be sent; you will see.

Uncaught Error: Message length exceeded maximum allowed length.

In my tests, the max message size is slightly above 32Mb, which is not enough for some large images, especially in a serialized state.

The Solution

Let's divide the message into chunks and send them in a separate sendMessage calls.

For the sender, I will be creating the sendChunkedMessage(message: any): Promise<any> async function that mimics the sendMessage signature and hides chunking and transmitting under the hood.

The original message is serialized and split into chunks. A group of messages is created to send individual chunks that share the generated requestId. The last message in the group contains done: true that signals the receiver that transmission is done.

// To filter out chunked messages on receiver side
const CHUNKED_MESSAGE_FLAG = 'CHUNKED_MESSAGE_FLAG'
const MAX_CHUNK_SIZE = 32 * 1024 * 1024; // 32Mb

const sendMessage = (message) =>
    new Promise(resolve =>
        chrome.runtime.sendMessage(message, response => {
            resolve(response);
        });
    );

const sendChunkedMessage = (message) => {
    // Generating requestId for the message
    const requestId = self.crypto.randomUUID();
    const messageSerialized = JSON.stringify(message);
    const len = messageSerialized.length;
    const step = MAX_CHUNK_SIZE;
    let ii = 0;

    // Sending messageSerialized in chunks in separate sendMessage calls
    while (ii < len) {
        const nextIndex = Math.min(ii + step, len);
        const substr = messageSerialized.substring(ii, nextIndex);

        await sendMessage({
            [CHUNKED_MESSAGE_FLAG]: true,
            requestId,
            chunk: substr
        });
        ii = nextIndex;
    }
    // At least 2 messages will be sent. Last one - with done: true
    const response = await sendMessage({
        [CHUNKED_MESSAGE_FLAG]: true,
        requestId,
        done: true
    });
}

On the receiver end, we need to create a handler that will be reconstructing messages from chunks based on requestId. I've created a function addOnChunkedMessageListener(handler: (request, sender, sendReponse) => boolean) that mimics chrome.runtime.onMessage.addListener signature hides implementation details under the hood.

Received chunks are stored into requestsStorage hashmap by requestId. When the done: true message is received, the original message is reconstructed by combining chunks from requestsStorage[requestId].

Then, provided handler function is executed with a reconstructed message

const requestsStorage = {};

const addOnChunkedMessageListener = (handler) => {
    const newListener = (request, sender, sendResponse) => {
        if (request && request[CHUNKED_MESSAGE_FLAG] && request.requestId) {
            const requestId = request.requestId;

            if (request.done) {
                const fullMessageSearialized = ''.concat.apply(
                    '',
                    requestsStorage[requestId]
                );
                delete requestsStorage[requestId];
                const fullMessage = JSON.parse(fullMessageSearialized);

                // async sendResponse can be enabled inside handler function
                return handler(fullMessage, sender, sendResponse);
            } else {
                if (!requestsStorage[requestId]) {
                    requestsStorage[requestId] = [];
                }
                requestsStorage[requestId].push(request.chunk);
                sendResponse({
                    status: 'PENDING'
                });
            }

            return false; // sync listener
        }
    }
    chrome.runtime.onMessage.addListener(newListener);
    return newListener;
};

Example Usage

content script:

sendChunkedMessage(largeMessage)
    .then(response => {
        ...
    })

background service worker:

addOnChunkedMessageListener((message, sender, sendResponse) => {
    // message === largeMessage
    ...
})

Large Response

The above solution works when we need to send a large message and receive a "normal size" response. But how can we send back the large response on the receiver side with sendResponse?

The idea is to send back an indication large response will follow, and temporarily add the same addOnChunkedMessageListener on the sender side, receive a chunked response, then remove the temporary listener.

To send a large response, sendChunkedResponse function should be used on the receiver side:

const sendChunkedResponse = ({ sendMessageFn } = {}) => (
    response,
    sendResponse,
) => {
    const requestId = self.crypto.randomUUID();
    // Sending an indication that file will be sent as chunked messages
    sendResponse({
        [CHUNKED_MESSAGE_FLAG]: true,
        requestId
    });
    // At this point content script has added a listener with addOnMessageWithChunksListener
    // Sending file contents as chunked messages
    sendChunkedMessage(response, {
        sendMessageFn: sendMessageFn || sendMessage,
        requestId
    });
};

I've modified sendChunkedMessage function with adding options: the ability to override the sendMessage function, override requestId, and support for receiving chunked responses.

const sendChunkedMessage = async (
    message,
    { sendMessageFn, requestId: requestIdOverriden } = {}
) => {
    const sendMessage = sendMessageFn || sendMessage;
    // Generating requestId for the message
    const requestId = requestIdOverriden || self.crypto.randomUUID();
    ...

    // If response indicates there will be a chunk message sent, adding a listener to retrieve full response
    if (response && response[CHUNKED_MESSAGE_FLAG]) {
        let listener;
        try {
            const fullResponse = await new Promise(resolve => {
                listener = addOnChunkedMessageListener(
                    (fullResponseFromListener, _, sendResponse) => {
                        sendResponse();
                        resolve(fullResponseFromListener);
                    },
                    {
                        requestIdToMonitor: response.requestId
                    }
                );
            });
            return fullResponse;
        } finally {
            if (listener) {
                removeOnChunkedMessageListener(listener);
            }
        }
    } else {
        return response;
    }
}

You may notice addOnChunkedMessageListener is also slightly modified to filter out incoming requestId with requestIdToMonitor option.

Example Usage With Large Response

content script - same:

sendChunkedMessage(largeMessage)
    .then(response => {
        ...
    })

background service worker:

addOnChunkedMessageListener((message, sender, sendResponse) => {
    // message === largeMessage
    ...
    sendChunkedResponse({
        sendMessageFn: message =>
            chrome.tabs.sendMessage(sender.tab.id, message)
    })(largeResponse, sendResponse);

    return true; // async listener
})

Conclusion

The solution was created to overcome message length limitation for Chrome Extensions messaging in Manifest V3; the solution works and I am using it in a real project. Happy to hear advice on how to make it better, and feel free to contribute! Code with an example extension can be found on GitHub: https://github.com/abelozerov/ext-send-chunked-message

Published to NPM. It can be installed with npm i ext-send-chunked-message

Thank you for reading!