Part 1:Building Your First Video Pipeline: FFmpeg & MediaMTX Basics

Introduction Imagine this: You're tasked with building a web application that displays live video feeds from security cameras, IoT devices, or webcams. Simple enough, right? You quickly discover that while HTML5 video works great for pre-recorded content, the world of real-time streaming is a maze of protocols, codecs, and compatibility issues that can turn a very "simple" feature into weeks of frustration. The core problem is interestingly simple: cameras speak RTSP, but browsers don't. Most IP cameras and professional video equipment use RTSP (Real Time Streaming Protocol) to broadcast their feeds. That's because it's reliable, has low latency, and is perfect for direct connections. But when you try to display that same stream in a web browser, you hit a wall. Modern browsers have dropped RTSP support entirely for security reasons since around 2010-2015, leaving developers scrambling for solutions. cameras speak RTSP, but browsers don't This is where the magic happens. This is where the magic happens. FFmpeg, the Swiss Army knife of video processing, and MediaMTX, a modern streaming server that acts as a universal translator between different video protocols. Together, they form the backbone of countless video applications, from Netflix's encoding pipeline to your local security system's web interface. FFmpeg MediaMTX In this series, we are going to build a complete, production-ready video streaming pipeline from the ground up. By the end of this first article, you'll have a live webcam feed streaming directly in your browser with remarkably low latency. Let's dive in. The Tools of the Trade FFmpeg: The Universal Video Swiss Army Knife FFmpeg is arguably one of the most important pieces of software you've never heard of. It powers video processing in applications ranging from VLC media player to professional broadcast systems. At its core, FFmpeg is a command-line tool that can read virtually any video or audio format and convert it to virtually any other format. The FFmpeg workflow follows a predictable pattern: Demuxing: Separating video and audio streams from container formats Decoding: Converting compressed data into raw video frames and audio samples Filtering: Applying transformations like scaling, cropping, or color correction Encoding: Compressing the processed data using codecs like H.264 or VP9 Muxing: Packaging the encoded streams into output containers Demuxing: Separating video and audio streams from container formats Demuxing: Decoding: Converting compressed data into raw video frames and audio samples Decoding: Filtering: Applying transformations like scaling, cropping, or color correction Filtering: Encoding: Compressing the processed data using codecs like H.264 or VP9 Encoding: Muxing: Packaging the encoded streams into output containers Muxing: For our streaming pipeline, FFmpeg will serve as the ingestion engine. It will capture video from sources like webcams or files, encode it efficiently, and push it to our streaming server using protocols like RTSP or RTMP (Real-Time Messaging Protocol). MediaMTX: The Modern Streaming Gateway While FFmpeg excels at processing video, it cannot serve multiple clients simultaneously. That's where MediaMTX comes in. MediaMTX is a modern, lightweight streaming server that acts as a universal media gateway. Think of MediaMTX as a protocol translator and traffic manager: It accepts incoming streams via RTSP, RTMP, WebRTC, or HLS It can re-package those streams into different formats for different clients It handles authentication, load balancing, and client management Most importantly for web developers, it can serve RTSP streams as WebRTC, making them accessible to browsers It accepts incoming streams via RTSP, RTMP, WebRTC, or HLS It can re-package those streams into different formats for different clients It handles authentication, load balancing, and client management Most importantly for web developers, it can serve RTSP streams as WebRTC, making them accessible to browsers The beauty of MediaMTX lies in its simplicity. A single binary with a YAML configuration file can handle complex streaming scenarios that would require multiple specialized servers in traditional setups. Hands-On Setup Before we start building, let's get our tools installed. Installing FFmpeg As of the time of writing this article, the latest ffmpeg release is v8.0, and that is what I will be using. As of the time of writing this article, the latest ffmpeg release is v8.0, and that is what I will be using. On Ubuntu/Debian: sudo apt-update sudo apt install autoconf automake build-essential pkg-config libx264-dev libvpx-dev libfdk-aac-dev git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg cd ffmpeg ./configure --enable-gpl --enable-libx264 --enable-nonfree make -j$(nproc) make install sudo apt-update sudo apt install autoconf automake build-essential pkg-config libx264-dev libvpx-dev libfdk-aac-dev git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg cd ffmpeg ./configure --enable-gpl --enable-libx264 --enable-nonfree make -j$(nproc) make install On MacOS (using Homebrew): brew install ffmpeg brew install ffmpeg On Windows: Download FFmpeg: Download FFmpeg: Go to https://ffmpeg.org/download.html Click on "Windows" Choose the gyan.dev build (recommended) Download the latest release version Go to https://ffmpeg.org/download.html https://ffmpeg.org/download.html Click on "Windows" Choose the gyan.dev build (recommended) Download the latest release version Extract the Files: Extract the Files: Extract the downloaded zip file to C:\ffmpeg You should see folders like bin, doc, etc. Extract the downloaded zip file to C:\ffmpeg C:\ffmpeg You should see folders like bin, doc, etc. bin doc Add to System PATH: Add to System PATH: Press Windows + R, type sysdm.cpl, press Enter Click Advanced tab → Environment Variables Under System Variables, find and select Path Click Edit → New Add: C:\ffmpeg\bin Click OK on all windows Press Windows + R, type sysdm.cpl, press Enter Windows + sysdm.cpl Click Advanced tab → Environment Variables Advanced Environment Variables Under System Variables, find and select Path System Variables Path Click Edit → New Edit New Add: C:\ffmpeg\bin C:\ffmpeg\bin Click OK on all windows OK Verify Installation: Verify Installation: ffmpeg -version ffmpeg -version You should see version information and a list of supported codecs and formats. Installing MediaMTX As of the time of writing this article, the latest ffmpeg release is v1.15.0, and that is what I will be using. As of the time of writing this article, the latest ffmpeg release is v1.15.0, and that is what I will be using. MediaMTX distributes as a single binary, making installation straightforward: MacOS and Linux Installation: # Download and extract (replace 'linux' with 'darwin' for MacOS) wget https://github.com/bluenviron/mediamtx/releases/latest/download/mediamtx_v1.15.0_linux_amd64.tar.gz tar -xzf mediamtx_v1.15.0_linux_amd64.tar.gz # Make executable and test chmod +x mediamtx # Add to system PATH sudo mv mediamtx /usr/local/bin/ # Run it to confirm installation mediamtx # Download and extract (replace 'linux' with 'darwin' for MacOS) wget https://github.com/bluenviron/mediamtx/releases/latest/download/mediamtx_v1.15.0_linux_amd64.tar.gz tar -xzf mediamtx_v1.15.0_linux_amd64.tar.gz # Make executable and test chmod +x mediamtx # Add to system PATH sudo mv mediamtx /usr/local/bin/ # Run it to confirm installation mediamtx Windows Installation: # Download and extract curl -L -O https://github.com/bluenviron/mediamtx/releases/latest/download/mediamtx_v1.15.0_windows_amd64.tar.gz # Extract to mediamtx_v1.15.0_windows_amd64 folder # Test run cd mediamtx_v1.15.0_windows_amd64 ./mediamtx.exe # Add to PATH mkdir C:\Users\{YOUR_USER}\bin Move-Item .\mediamtx.exe C:\Users\{YOUR_USER}\bin\ # Download and extract curl -L -O https://github.com/bluenviron/mediamtx/releases/latest/download/mediamtx_v1.15.0_windows_amd64.tar.gz # Extract to mediamtx_v1.15.0_windows_amd64 folder # Test run cd mediamtx_v1.15.0_windows_amd64 ./mediamtx.exe # Add to PATH mkdir C:\Users\{YOUR_USER}\bin Move-Item .\mediamtx.exe C:\Users\{YOUR_USER}\bin\ Add to Windows PATH: Windows + R → sysdm.cpl → Advanced → Environment Variables → System Variables → Path → Edit → New → Add C:\Users\{YOUR_USER}\bin Add to Windows PATH sysdm.cpl C:\Users\{YOUR_USER}\bin Note: Windows Defender may flag the download - temporarily disable if needed. Note: When you run MediaMTX for the first time, you'll see output like: MediaMTX is now running and ready to accept streams. Project 1: Streaming a Video File Let's start simple by streaming a video file. This simulates a live source and helps us understand the basic pipeline without the complexity of hardware interfaces. First, create a basic MediaMTX configuration file. Create mediamtx.yml: mediamtx.yml # Basic MediaMTX configuration paths: test_video: source: publisher # Basic MediaMTX configuration paths: test_video: source: publisher This configuration creates a path called test_video that accepts published streams from any source. test_video Run mediamtx with the config file you created: # In the directory you created the mediamtx.yml mediamtx mediamtx.yml # In the directory you created the mediamtx.yml mediamtx mediamtx.yml Now, let's use FFmpeg to stream a video file to MediaMTX. You'll need a video file for testing. Any MP4, AVI, or MOV file will work: ffmpeg -re -i your_video.mp4 -c:v libx264 -preset fast -c:a aac -f rtsp rtsp://localhost:8554/test_video ffmpeg -re -i your_video.mp4 -c:v libx264 -preset fast -c:a aac -f rtsp rtsp://localhost:8554/test_video Let's break down this command: -re: Read input at its native frame rate (essential for live streaming) -i your_video.mp4: Input file -c:v libx264: Use H.264 video codec (widely compatible) -preset fast: Encoding speed vs. compression trade-off -c:a aac: Use AAC audio codec -f rtsp: Output format is RTSP rtsp://localhost:8554/test_video: Destination URL -re: Read input at its native frame rate (essential for live streaming) -re -i your_video.mp4: Input file -i your_video.mp4 -c:v libx264: Use H.264 video codec (widely compatible) -c:v libx264 -preset fast: Encoding speed vs. compression trade-off -preset fast -c:a aac: Use AAC audio codec -c:a aac -f rtsp: Output format is RTSP -f rtsp rtsp://localhost:8554/test_video: Destination URL rtsp://localhost:8554/test_video If everything works correctly, you'll see FFmpeg output showing frame processing statistics. Testing Your Stream Open VLC Media Player and: Go to Media → Open Network Stream Enter: rtsp://localhost:8554/test_video Click Play Go to Media → Open Network Stream Enter: rtsp://localhost:8554/test_video rtsp://localhost:8554/test_video Click Play You should see your video playing! This confirms that MediaMTX is receiving the stream from FFmpeg and serving it via RTSP. Project 2: Streaming a Webcam Now, let's capture something live. We'll stream directly from your webcam to create a real-time video feed. First, let's add a new path in our mediamtx config: # Basic MediaMTX configuration paths: test_video: source: publisher webcam: source: publisher # Basic MediaMTX configuration paths: test_video: source: publisher webcam: source: publisher Now we need to identify your webcam device: On Linux: # List video devices ls /dev/video* # Usually /dev/video0 for the first webcam # List video devices ls /dev/video* # Usually /dev/video0 for the first webcam On MacOS: # List available devices ffmpeg -f avfoundation -list_devices true -i "" # List available devices ffmpeg -f avfoundation -list_devices true -i "" On Windows: # List available devices ffmpeg -list_devices true -f dshow -i dummy # OR Get-PnpDevice -Class Camera | Select-Object FriendlyName, Status # List available devices ffmpeg -list_devices true -f dshow -i dummy # OR Get-PnpDevice -Class Camera | Select-Object FriendlyName, Status You'll see output listing available cameras and microphones. Note the device index (usually 0 for the built-in camera). Now, let's stream your webcam: On Windows: ffmpeg -f dshow -rtbufsize 100M -i video="Integrated Webcam" -c:v libx264 -preset ultrafast -tune zerolatency -f rtsp rtsp://localhost:8554/webcam ffmpeg -f dshow -rtbufsize 100M -i video="Integrated Webcam" -c:v libx264 -preset ultrafast -tune zerolatency -f rtsp rtsp://localhost:8554/webcam Let's break down this command: Input Parameters: Input Parameters: -f dshow - Use DirectShow input format (Windows-specific for cameras/microphones) -rtbufsize 100M - Set real-time buffer size to 100MB (prevents dropped frames) -i video="Integrated Webcam" - Input source: video device named "Integrated Webcam" -f dshow - Use DirectShow input format (Windows-specific for cameras/microphones) -f dshow -rtbufsize 100M - Set real-time buffer size to 100MB (prevents dropped frames) -rtbufsize 100M -i video="Integrated Webcam" - Input source: video device named "Integrated Webcam" -i video="Integrated Webcam" Output Parameters: Output Parameters: -c:v libx264 - Video codec: use H.264 encoder -preset ultrafast - Encoding preset: prioritize speed over compression quality -tune zerolatency - Optimize encoding for real-time streaming (minimal buffering) -f rtsp - Output format: Real Time Streaming Protocol rtsp://localhost:8554/webcam - Output destination: local RTSP server on port 8554, path "/webcam" -c:v libx264 - Video codec: use H.264 encoder -c:v libx264 -preset ultrafast - Encoding preset: prioritize speed over compression quality -preset ultrafast -tune zerolatency - Optimize encoding for real-time streaming (minimal buffering) -tune zerolatency -f rtsp - Output format: Real Time Streaming Protocol -f rtsp rtsp://localhost:8554/webcam - Output destination: local RTSP server on port 8554, path "/webcam" rtsp://localhost:8554/webcam On MacOS: ffmpeg -f avfoundation -framerate 30 -video_size 1280x720 -i "0" -c:v libx264 -preset ultrafast -tune zerolatency -f rtsp rtsp://localhost:8554/webcam ffmpeg -f avfoundation -framerate 30 -video_size 1280x720 -i "0" -c:v libx264 -preset ultrafast -tune zerolatency -f rtsp rtsp://localhost:8554/webcam Input Parameters: Input Parameters: -f avfoundation - Use AVFoundation input format (macOS-specific for cameras/microphones) -framerate 30 - Capture at 30 frames per second -video_size 1280x720 - Set capture resolution to 1280x720 (720p) -i "0" - Input source: device index 0 (first camera device) -f avfoundation - Use AVFoundation input format (macOS-specific for cameras/microphones) -f avfoundation -framerate 30 - Capture at 30 frames per second -framerate 30 -video_size 1280x720 - Set capture resolution to 1280x720 (720p) -video_size 1280x720 -i "0" - Input source: device index 0 (first camera device) -i "0" Output Parameters: (same as Windows) Output Parameters: -c:v libx264 - Video codec: H.264 encoder -preset ultrafast - Fastest encoding preset -tune zerolatency - Real-time streaming optimization -f rtsp - RTSP output format rtsp://localhost:8554/webcam - RTSP server destination -c:v libx264 - Video codec: H.264 encoder -c:v libx264 -preset ultrafast - Fastest encoding preset -preset ultrafast -tune zerolatency - Real-time streaming optimization -tune zerolatency -f rtsp - RTSP output format -f rtsp rtsp://localhost:8554/webcam - RTSP server destination rtsp://localhost:8554/webcam On Linux: ffmpeg -f v4l2 -i /dev/video0 -c:v libx264 -preset ultrafast -tune zerolatency -c:a aac -f rtsp rtsp://localhost:8554/webcam ffmpeg -f v4l2 -i /dev/video0 -c:v libx264 -preset ultrafast -tune zerolatency -c:a aac -f rtsp rtsp://localhost:8554/webcam Input Parameters: Input Parameters: -f v4l2 - Use Video4Linux2 input format (Linux-specific for cameras) -i /dev/video0 - Input source: first video device in Linux (/dev/video0) -f v4l2 - Use Video4Linux2 input format (Linux-specific for cameras) -f v4l2 -i /dev/video0 - Input source: first video device in Linux (/dev/video0) -i /dev/video0 Output Parameters: Output Parameters: -c:v libx264 - Video codec: H.264 encoder -preset ultrafast - Fastest encoding preset -tune zerolatency - Real-time streaming optimization -c:a aac - Audio codec: Advanced Audio Coding (AAC) -f rtsp - RTSP output format rtsp://localhost:8554/webcam - RTSP server destination -c:v libx264 - Video codec: H.264 encoder -c:v libx264 -preset ultrafast - Fastest encoding preset -preset ultrafast -tune zerolatency - Real-time streaming optimization -tune zerolatency -c:a aac - Audio codec: Advanced Audio Coding (AAC) -c:a aac -f rtsp - RTSP output format -f rtsp rtsp://localhost:8554/webcam - RTSP server destination rtsp://localhost:8554/webcam Note: We're using the path /webcam instead of /test_video. Test this stream in VLC using rtsp://localhost:8554/webcam. You should see your live webcam feed with minimal delay! Note: /webcam /test_video rtsp://localhost:8554/webcam Project 3: The Magic of WebRTC Here's where things get exciting. While RTSP works great for applications like VLC, it won't work in web browsers. But MediaMTX has a superpower: it can automatically convert RTSP streams to WebRTC, which browsers understand perfectly. Let's enable WebRTC in MediaMTX. Update your mediamtx.yml configuration: mediamtx.yml # Basic MediaMTX configuration webrtc: yes webrtcAddress: :8889 webrtcEncryption: no webrtcAllowOrigin: '*' webrtcLocalUDPAddress: :8189 webrtcIPsFromInterfaces: yes paths: test_video: source: publisher webcam: source: publisher # Basic MediaMTX configuration webrtc: yes webrtcAddress: :8889 webrtcEncryption: no webrtcAllowOrigin: '*' webrtcLocalUDPAddress: :8189 webrtcIPsFromInterfaces: yes paths: test_video: source: publisher webcam: source: publisher Restart MediaMTX with this new configuration and keep your webcam stream running with FFmpeg. Open your browser to http://localhost:8889/webcam. Your webcam feed should start playing directly in the browser. http://localhost:8889/webcam This is the magic moment. You've just built a complete real-time video pipeline that captures live video, processes it, and delivers it to web browsers using modern WebRTC technology. The same architecture powers professional applications serving thousands of concurrent viewers. Understanding What Just Happened Let's trace the complete data flow: Your webcam produces raw video frames FFmpeg captures these frames, encodes them as H.264, and streams them via RTSP to MediaMTX MediaMTX receives the RTSP stream and makes it available on the /webcam path When a browser requests the stream via WebRTC, MediaMTX automatically converts the RTSP stream to WebRTC format The browser receives WebRTC packets and renders them in real-time Your webcam produces raw video frames FFmpeg captures these frames, encodes them as H.264, and streams them via RTSP to MediaMTX MediaMTX receives the RTSP stream and makes it available on the /webcam path /webcam When a browser requests the stream via WebRTC, MediaMTX automatically converts the RTSP stream to WebRTC format The browser receives WebRTC packets and renders them in real-time This pipeline is remarkably efficient because MediaMTX doesn't re-encode the video. It simply repackages the H.264 stream into different container formats for different protocols. Conclusion & What's Next You've just built a foundational real-time video streaming pipeline. Let's recap what you've accomplished: Installed and configured FFmpeg and MediaMTX Streamed a pre-recorded video file to simulate live content Captured and streamed live video from your webcam Enabled browser-based viewing using WebRTC Created a low-latency video pipeline suitable for interactive applications Installed and configured FFmpeg and MediaMTX Streamed a pre-recorded video file to simulate live content Captured and streamed live video from your webcam Enabled browser-based viewing using WebRTC Created a low-latency video pipeline suitable for interactive applications This setup demonstrates the core concepts that power much larger streaming systems. The same pattern (capture, encode, serve) scales from single streams to thousands of concurrent feeds. However, our current setup has some limitations that prevent it from being production-ready: No authentication or security measures Only works on localhost Can't handle real-world video sources like IP cameras No monitoring or error handling No authentication or security measures Only works on localhost Can't handle real-world video sources like IP cameras No monitoring or error handling In Part 2, we'll address these challenges by securing our pipeline, connecting to real IP cameras, and preparing our system for internet deployment. We'll explore authentication mechanisms, handle diverse video formats, and transform our localhost demo into a robust, secure streaming service. The journey from "works on my machine" to "production-ready" is where the real engineering challenges begin and where the most interesting solutions emerge. Ready to take your streaming pipeline to the next level? Continue with Part 2: Beyond Localhost: Security, Authentication, and Real-World Sources where we'll secure our setup and connect to real-world video sources. Part 2: Beyond Localhost: Security, Authentication, and Real-World Sources