WebTorrent Trackers improved memory usage by 16x
Intro
I’ve been running the first community WebTorrent tracker (tracker.btorrent.xyz) server for a while now. The WebRTC network has been steadily growing and right now there’s at least 700 connected peers at any given time (See tracker’s stats).
Every time there’s a popular post or news release about WebTorrent and ideas/demos/products built around it, my tracker can’t handle the sudden non-collected allocation of RAM for the new websockets and it ‘dies’, even PM2 dies.
I’ve received donation offers from community users who want to support the server’s cost but I was holding that until the issues with the tracker were fully fixed; there were some horrible memory leaks and issues that were addressed and fixed this year that allowed a substantial increase in connected peers (less than 100 to now 800) in the same memory space of 400–500MB.
Hopefully donations won’t be needed any time soon as my tracker now runs with uWS, a not so known(18K dls/mo vs 6.6M dls/mo of ws) WebSocket server & client library that promises the best performance and up to 47x less memory usage than ws.
ws
Currently, bittorrent-tracker module uses ws WebSocket implementation to handle the incoming peers to be signaled for WebRTC connections. Using about 0.45MB per websocket connection, in JSArrayBufferData objects:
Heapshot; Sintel torrent (most popular tracked torrent) is selected
Switching to µWS
Switching to uws can be as simple as replacing ws with uws:
Just swap ‘ws’ with ‘uws’
But beware, accessing to the remote socket address is done differently and you will only be able to access the request’s headers in the connection event; uws nullifies upgradeReq after. Also, proxied websockets will have 127.0.0.1 as their remote address (Open issue), so you must access upgradeReq headers to get the real IP.
Results
A few minutes after the switch, all peers (800+) were connected back again and from 446MB of allocated memory, it jumped to under 30MB. That’s almost 1/16th of the memory!
PM2 Monitor