How to Create a Personal Residential Proxy to Bypass Geo Restrictions

Problem

Residential proxies have a lot of useful applications, such as web scraping, online research, bypassing geo-restrictions and anonymous browsing. But, they can be expensive, with some charging more than $100 a month for access to over 10 GB of data.

Last year, when I was travelling to India and other parts of Southeast Asia, I wanted a reliable way which would let me access content on various subscription services like Netflix and Amazon Prime without geo-restrictions. I wanted to do this while

1. Not having to pay more than $100 per month for VPN/residential proxy service. Also, Netflix would occasionally block me when using VPN.
2. Not paying an additional amount for a static IP for my home internet connection back in the US.

Solution

I discovered a solution by creating my own residential proxy on my home desktop in the US.

By setting up a peer-to-peer(P2P) VPN and connecting my laptop and desktop to it, I was able to route the traffic of my laptop in India to my desktop in the US, which then forwarded these requests to www/public internet before returning the response from my home network.

All this without having to go through the hassle of setting up connection forwarding configuration in my home router.

What is a P2P VPN

Peer-to-peer(P2P) is an architecture model for VPNs used to create a secure shared environment for users. A P2P based VPN differs from the standard hub-and-spoke model by not routing all traffic through a central server, avoiding congestion. This improves security and allows for more efficient scaling by adding nodes without overburdening the network. Additionally, connections in a P2P VPN utilize the quickest path, reducing delays.

There a multiple open-source, freemium and paid P2P VPN solutions available. Tinc is one of popular open-source VPN.

How does the network setup look?

The network stack consists of the following components:

1. Desktop at home (US)

2. My laptop (Outside US)

3. P2P vpn daemon running on both machine via same account

4. Http proxy server running on the desktop

   
   

The diagram below depicts the flow of information to retrieve a public webpage from my laptop outside of the US.

I wanted a P2P VPN which is easy to setup and requires near-zero config. I ended up using tailscale. The steps below are mostly generic for any VPN but you may find this most suitable if using tailscale.

Steps taken to setup this network

1. Setting up the vpn client on desktop and laptop:

   1. Download and install VPN client on desktop and laptop.

   2. If using a zero-config VPN, register your account with vpn and sign-in on both the machines with the same account

      
      

2. Setting up a forward http proxy server on desktop:

   You can use any open source proxy server. I used https://github.com/abhinavsingh/proxy.py as it was easy to set up. I ran it within a docker container. The docker image for this proxy is available on docker hub. To run it within a docker container you can execute this command

   docker run -it -p 8899:8899 --rm abhinavsingh/proxy.py:latest
   

   
   

   In the command above, the proxy server is listening on port 8899. This will be useful when we specify the url of proxy server in the applications within the laptop

   
   

3. Identify the ip address of desktop in the VPN network: After installing and signing into your VPN account, lets figure out the ip address of the desktop which will be used by the laptop to forward its http requests .

   
   

   For this, you can list the ip addresses for all the network interfaces on the desktop. Then identify the ip address for an interface whose name starts with the name of your VPN client, for my case, it starts with “tailscale”.

   
   

   I used the ip command on my linux machine’s terminal to identify the ip address

   $ ip a
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
       inet 127.0.0.1/8 scope host lo
          valid_lft forever preferred_lft forever
       inet6 ::1/128 scope host 
          valid_lft forever preferred_lft forever
   ….
   4: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qdisc fq_codel state UNKNOWN group default qlen 500
       link/none 
       inet 100.113.xxx.xx/32 scope global tailscale0
          valid_lft forever preferred_lft forever
    …
   

   Here I was able to find the ip address of desktop within my local tailscale network in the network interface tailscale0 : 100.113.xxx.xx

   
   

4. Using the address of proxy server on laptop to access public web:

   Now that we know the ip address of desktop (let's call it $ip_desktop) and the port number for the http proxy server on desktop (let’s call it $proxy_port), we can forward all the network traffic from laptop via proxy address http://$ip_desktop:$proxy_port.

   
   

   For example you can send a curl request to example.com from laptop via desktop like

   curl --proxy "http://$ip_desktop:$proxy_port" "https://example.com"
   

Using the proxy server

The above instructions demonstrate how to create a free residential proxy server while travelling using a spare computer at home. Now you can use this proxy address on your personal laptop in any of the following ways:

1. Within your application code inside the http client libraries of your web framework/language (requests in python, fetch in javascript etc)
2. For all the web requests from your laptop. This way you can forward any kind http request via desktop’s proxy server. In ubuntu you can do this by following these steps https://help.ubuntu.com/stable/ubuntu-help/net-proxy.html.en
3. Just in your browser. Although Chrome and Firefox don’t provide a native way to setup exclusive http proxy settings for them, you can do this via extensions like https://github.com/foxyproxy/firefox-extension

I chose 2. and forwarded all the http requests from my laptop to desktop. I used this setup for more than two months without any noticeable downtime.

Conclusion

In this article we went over how to setup a personal residential proxy using

1. P2P VPN
2. Two or more personal computers

Depending on the choice of the VPN, the whole setup can be done without any additional cost, assuming you already own multiple computers. Using this residential proxy you can then access the internet anonymously without any geo-restrictions due to your current physical location.