Frame Stream Player

A small web app that plays a remote video stream without using browser video decoding. The server uses ffmpeg to decode the input URL into:

• an MP3 audio stream served to a normal <audio> element
• timed JPEG image frames sent over a WebSocket and painted onto a <canvas>

This is meant for machines where image and audio decoding work but browser video decoding is unavailable or unreliable.

Run

npm install
npm start

Open http://localhost:3000, paste a direct HTTP(S) stream URL, and click Play. Use Queue to add a URL to Recents without starting playback.

Set LOCAL_VIDEOS=/path/to/videos before starting the server to enable Play Local. The picker lets you navigate directories under that root, search the library, and play selected files through the server-side ffmpeg pipeline as MP3 audio plus JPEG canvas frames.

You need an ffmpeg binary with decoders for the stream's video and audio codecs. YouTube page links also need yt-dlp; Docker installs it automatically from the upstream master branch at image build time. If your stream is H.264 or HEVC, make sure your installed ffmpeg actually includes those decoders. You can point the app at different binaries with:

FFMPEG_PATH=/path/to/ffmpeg npm start
YT_DLP_PATH=/path/to/yt-dlp npm start

Docker

docker build -t frame-stream-player .
docker run --rm -p 3000:3000 frame-stream-player

Then open http://localhost:3000.

For Docker Compose:

docker compose -f docker-compose-example.yml up --build

The app uses CPU decoding by default, so no video device is required. The compose example includes commented VAAPI/NVIDIA passthrough options for future hardware-accelerated ffmpeg setups, but hardware acceleration is usually only useful when server CPU is saturated.

Recently played URLs and favorites are stored globally by the backend. In Docker Compose, they are persisted in the frame-stream-data named volume.

To use local playback in Docker, mount a host video directory into the container and set LOCAL_VIDEOS to the container path, for example /app/local-videos.

ffmpeg worker lifecycle, stderr warnings/errors, and source proxy open/close events are written to stdout/stderr, so they appear in docker logs. For more detail while debugging a stream, set FFMPEG_LOG_LEVEL=info in Docker Compose and run:

docker logs -f frame-stream-player

The app sets FFMPEG_INPUT_SEEKABLE=0 by default so ffmpeg reads stream inputs sequentially and avoids extra HTTP range connections. If a specific VOD file requires seeking for metadata, set FFMPEG_INPUT_SEEKABLE=-1 to restore ffmpeg's automatic behavior. For ffmpeg-owned HTTP inputs, reconnect handling is enabled by default with FFMPEG_HTTP_RECONNECT=1. FFMPEG_HTTP_RECONNECT_MAX_RETRIES is applied only when the installed ffmpeg supports that HTTP protocol option.

YouTube URLs are resolved server-side with yt-dlp before they enter the existing ffmpeg pipeline. Recents and favorites keep the original YouTube URL, while the short-lived playback session uses the resolved media URL and headers returned by yt-dlp. Tune the selected format with YT_DLP_FORMAT and the resolver timeout with YT_DLP_TIMEOUT_MS.

JPEG frames are dropped when the browser WebSocket falls behind instead of letting stale frames queue indefinitely. Tune the server-side backlog cap with MAX_WS_BUFFER_BYTES; the default is 2097152.

In single mode, audio output from ffmpeg is buffered before it is written to the browser so short HTTP backpressure pauses are less likely to stall frame generation. Tune the cap with MAX_AUDIO_QUEUE_BYTES; the default is 4194304.

Playback uses PLAYBACK_CONNECTION_MODE=split by default. The Docker Compose example sets PLAYBACK_CONNECTION_MODE=relay so IPTV-style streams can be tested with one upstream connection.

Available playback modes:

• split: Separate source connections and separate ffmpeg workers for audio and JPEG frames. This is usually the smoothest mode.
• relay: One source connection from the backend, then the compressed input bytes are teed into separate audio and frame ffmpeg workers. This is intended for IPTV hosts that stop early or reject multiple active connections.
• single: One source connection and one ffmpeg worker with both audio and frame outputs. This is the simplest one-connection fallback, but audio and frame delivery can affect each other.

Finite-duration streams are treated as recorded video and become seekable once metadata is available. Metadata probing is enabled by default except in relay, where METADATA_PROBE_ENABLED=0 avoids extra upstream connections. Set METADATA_PROBE_ENABLED=1 if you want recorded relay sessions to become seekable and switch to the seek-capable split path.

Relay mode uses bounded per-worker input queues so one branch can briefly lag without immediately stalling the other. Tune the cap with MAX_RELAY_BRANCH_QUEUE_BYTES; the default is 8388608.

Tuning

The UI intentionally hides these settings, but the backend still supports them through POST /api/session.

• Frame rate defaults to 24fps. Lower it if the client cannot keep up.
• Max width defaults to 960px, and the client now caps each session to its viewport width. Lower DEFAULT_FRAME_WIDTH first if bandwidth or image decode is the bottleneck.
• JPEG quality uses ffmpeg's -q:v scale, where lower is better/larger. Set the default with JPEG_QUALITY; 7 is the fallback, 2 is high quality, and 18 is rough but much lighter.
• Audio defaults to stereo MP3 at 160k. Tune with DEFAULT_AUDIO_BITRATE, DEFAULT_AUDIO_CHANNELS, and DEFAULT_AUDIO_SAMPLE_RATE.

Tradeoffs

JPEG frames are used instead of PNG or GIF. PNG is usually too large for 24fps video, and GIF has poor quality and weak timing control. JPEG is simple, browser-native, streamable per frame, and lets the audio element act as the playback clock.

The default split mode starts separate ffmpeg workers for audio and frames. That is simple and usually smoother for direct files and many HTTP streams, but live streams can have small startup offset differences and some hosts only allow one active connection. Relay mode avoids that host-side issue while keeping separate audio/frame workers, but it works best with sequential stream containers such as MPEG-TS. Single mode is kept as a fallback. The input URL is proxied or relayed by the backend before it is handed to ffmpeg, so query-string tokens are not exposed in ffmpeg process arguments.

Arbitrary URLs are still fetched by your server, so do not expose this app publicly without adding authentication and URL allowlisting.