505 reads

Follow These Steps To Replay Browser’s Network Offline

by Andrew MillenMay 22nd, 2021

Too Long; Didn't Read

This short tutorial explains how to replay the captured browser’s network offline using Python. The HAR file is used to capture requests which exist in the HAR file. The proxy server listens on:127.0.1:8080. The MITM proxy is set up in order to intercept certain requests from the HAR. In order to test the HAR replay you have to make the browser go through the proxy server, e.g. for Chrome the command would be: HTTPS://://://www.com/endgame-is-on/HAReplay.

Companies Mentioned

Coin Mentioned

featured image - Follow These Steps To Replay Browser’s Network Offline

This short tutorial explains how to replay the captured browser’s network offline using Python.

Once you have captured the browser’s network as a HAR file, you can set up a proxy server in order to provide the responses for certain requests which exist in the HAR.

Let’s start with creating a dictionary from the HAR.

import json
dict_url_response = {}
fob = open("har/cap1.har", "r")
data = json.load(fob)
fob.close()
entries = data["log"]["entries"]
for entry in entries:
    url = entry["request"]["url"]
    response = entry["response"]
    dict_url_response[url] = response

Now create a class which we will be using for interception in the proxy.

from mitmproxy.net.http import Response, Headers
class Interception:
    def request(self, flow):
        try:
            url = flow.request.url
            har_response = dict_url_response[url]
            text = har_response["content"]["text"]
            byt = str.encode(text)
            list_headers = []
            for obj in har_response["headers"]:
                list_headers.append((str.encode(obj["name"]), str.encode(obj["value"])))
            headers = Headers(list_headers)
            response = Response(
                http_version = str.encode(har_response["httpVersion"]),
                status_code = har_response["status"],
                reason = str.encode(har_response["statusText"]),
                headers = headers,
                content = byt,
                trailers = None,
                timestamp_start = 0.,
                timestamp_end = 1.
            )

        except KeyError:
            print("Request URL not found in HAR")
            headers = Headers([
                (b'Content-Type', b'text/plain; charset=utf-8'),
            ])
            response = Response(
                http_version = str.encode("http/2.0"),
                status_code = 200,
                reason = str.encode("OK"),
                headers = headers,
                content = str.encode("Request URL not found in HAR\n"),
                trailers = None,
                timestamp_start = 0.,
                timestamp_end = 1.
            )
        flow.response = response

Finally, we just have to start the MITM proxy with the interception.

from mitmproxy.options import Options
from mitmproxy.proxy.config import ProxyConfig
from mitmproxy.proxy.server import ProxyServer
from mitmproxy.tools.dump import DumpMaster
import threading
import asyncio
intercept = Interception()
def thread_func(loop, m):
    asyncio.set_event_loop(loop)
    m.run_loop(loop.run_forever)
options = Options(
    listen_host='0.0.0.0',
    listen_port=8080,
    http2=True
)
m = DumpMaster(
    options,
    with_termlog=True,
    with_dumper=True
)
m.addons.add(intercept)
config = ProxyConfig(options)
m.server = ProxyServer(config)
loop = asyncio.get_event_loop()
t = threading.Thread(
    target=thread_func,
    args=(loop, m)
)
t.start()

The

thread_func

is used in order to keep the proxy server running. The proxy server listens on

127.0.0.1:8080

In order to test the HAR replay you have to make the browser go through the proxy server, e.g. for Chrome the command would be:

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --proxy-server=127.0.0.1:8080

Sometimes the requests that are captured in the HAR are gzip or brotli compressed. In this GitHub repository such cases are taken into account:

https://github.com/endgame-is-on/HAReplay

Also published on Medium's endgame-is-on

L O A D I N G
. . . comments & more!

About Author

Andrew Millen@endgame

Senior Software Developer

Read my stories About @endgame

TOPICS

programming #python #mitm #proxy #network #chrome #proxy-servers #python3 #browser

THIS ARTICLE WAS FEATURED IN...