Twitch Enhanced Broadcasting changes everything about streaming to Twitch

At CES 2024, Nvidia and Twitch announced a collaboration with OBS called “Twitch Enhanced Broadcasting” — a complete overhaul of the traditional streamer-to-Twitch pipeline and encoding experience.

This new feature aims to make streaming easier, low-latency and buffer-free viewing more accessible, and has the potential to revolutionize video quality on Twitch along the way. And yes, AV1 is coming to Twitch, too — though maybe not as immediately as many are hoping for.

Twitch Enhanced Broadcasting is a new process for streaming from OBS Studio to Twitch that is entering beta this week. Enhanced Broadcasting allows the streamer to supplement Twitch’s stream transcoding (converting the original live stream, sent by the streamer, into lower resolution and lower bitrate versions) on their own hardware and send multiple different quality versions of their stream to Twitch at the same time. The test phase involves a new beta build of OBS Studio and can be signed up for via streamers’ Twitch dashboards.

Two major changes will be included with this beta:

1. Multitrack streaming to Twitch
2. Automatic configuration of stream settings

Multiple video tracks

The first major change comes with a new revision to the Enhanced RTMP streaming protocol (which was introduced in early 2023 to accommodate AV1 and HEVC live streaming) that allows for multiple video tracks to be packaged in the same stream. While FFMPEG and other video encoding tools have been able to leverage multiple video tracks in certain file containers before, this is the first time traditional RTMP streaming has been capable of such complexity. Previously, OBS users have been able to leverage a second audio track for Twitch’s “VOD Track” feature: This update works similarly, but for video instead of audio.

Multitrack video will be used to supplement Twitch’s transcoding pipeline, by having the streamer’s computer encode the stream at multiple different resolutions and frame rates and send all of those different video tracks to Twitch in one RTMP stream, to be passed on to the viewer.

During the initial beta phase, Twitch aims to target up to three simultaneous streams at a time (such as 1080p60 Source, 720p, and 360p) — but Nvidia stated that the goal is to reach a total of five concurrent streams (the driver-enforced encoding session limit for consumer GeForce GPUs) to cover a wider gamut of formats, and include higher resolution formats such as 1440p and 4K in the future.

Automatic settings configuration

Enabling Enhanced Broadcast in this new OBS beta will also take away the stress and complication of optimizing streaming and encoder settings for Twitch. This new service will ping Twitch’s servers, analyze the streamer’s system specs, available resources, internet latency and bandwidth, then generate the best settings for their stream (including adjusting how many transcodes are generated by the streamer’s hardware).This both optimizes the encoding ladder for optimal results from end to end, but also just makes streaming a lot more accessible. Once this feature is rolled out, there will be no more reason to tweak encoder flags or stress about settings.

Why streamers should care

While it’s easy to speculate that a significant motivator for this new feature roll-out is cost savings for Twitch, given the ongoing discussions of Twitch’s costs when it comes to improving the revenue split with creators, Twitch’s exit from South Korea, and the next round of layoffs that hit this week — there are some immediate benefits that streamers, and more importantly viewers, will see from using this feature.

• Reduced latency: When streams are transcoded by Twitch’s servers, additional latency is added to the broadcast due the processing time required. When streamers handle the transcoding step themselves, all Twitch must do is relay that feed to the viewer, reducing the latency and helping improve the “real time” feeling and bolster chat interaction. Twitch streams already have significantly lower latency than YouTube streams, but this will improve things even further.
• Quality accessibility: Stream quality and the accessibility of high quality streams is normalized across all streamers through this update, as streams will no longer be limited by the streamer’s intimate knowledge of encoder settings and optimization tips.
• Viewer accessibility: Additionally, the wider spread of transcoding on streams that Enhanced Broadcasting will allow means more streams than ever before can be enjoyed by mobile viewers or viewers with slower internet connections!
• Quality improvement: By encoding the lower resolution copies of the stream from the streamer’s raw OBS feed on their system, rather than from the already-compressed 1080p copy of the stream, lower resolution formats on Twitch have the potential to be significantly higher-quality than ever before, without requiring better connections.
• Cost reduction: Twitch will not have to allocate as many resources to transcoding workloads for streamers, and will not have to scale resources to try to provide transcoding to everyone, thus reducing costs to Twitch.

Twitch does not have the resources to roll out transcoding to everyone. Partners are guaranteed transcoding on their streams, and Affiliates get priority over everyone else, but many Affiliates and most normal streamers do not receive transcoding on their live streams. This paradigm requires many streamers to decide between higher-quality streams or reaching as many people as possible, as 6- to 8mbps 1080p60 streams just aren’t playable for many viewers at home. Enhanced Broadcasting will allow many more streamers to both have a high-quality stream and be more accessible to those viewers.

Into the future

The initial beta testing of Enhanced Broadcasting will be focused on evaluating the multitrack video streaming and making sure all this technology works and stays stable first and foremost. But eventually the feature will lead to some pretty cool new capabilities when streaming to Twitch.

To accommodate the multiple video tracks, the aggregate bitrate cap (total bitrate across all tracks) that streamers can send to Twitch with this feature will be increased to 10mbps. This is already nice, but it is possible that as the testing commences and stability is established, the cap could continue to increase and allow streamers to have even higher-quality streams for their original source format.

Quality increases will also come as the beta will include experimenting with enabling HEVC and AV1 video codecs for streaming. These are higher-efficiency video codecs that provide significantly higher quality streams for the same bitrate compared to H264 (the current codec supported by Twitch), or allow for the same quality to be had at a much lower bitrate, leaving room for more tracks. During my initial AV1 stream testing to YouTube, stream quality was impressive even at bitrates around 1mbps.

These newer codecs will also be a must for the newer 1440p and 4K formats that Nvidia teased during their announcement of Enhanced Broadcasting. Higher bitrates and more efficient codecs will also help make 120fps streaming a reality, too.

The multitrack video functionality also has potential to introduce other experiences like simultaneous landscape and portrait scene compositions, to accommodate mobile viewers even more — though Twitch has not announced any plans for such a feature just yet.

Requirements

Since this beta phase is a collaboration with Nvidia, streamers with Nvidia RTX graphics cards are being prioritized for entry into the beta. However, long-term Enhanced Broadcasting will be available for everyone — with the only requirement being to have some sort of GPU-accelerated or hardware-accelerated encoding available on your system. 

Streamers can sign up for the beta at https://dashboard.twitch.tv/broadcast. You do not need to be a Twitch Partner to apply — in fact, Partners aren’t a direct priority since they will continue to receive guaranteed transcoding through Twitch, anyway.

No one will be left behind

It’s easy to get lost in all the big talk about AV1 encoding (which is restricted to the latest generation of graphics cards), Nvidia sponsoring the beta, and multiple video tracks and worry that you’ll be left behind. Let me assure you that this will not be the case. The primary function of Enhanced Broadcasting is to increase accessibility across the board — everyone wins here.

Don’t have an RTX 40 series GPU for AV1 encoding or an RTX card at all? No big deal. This won’t stay Nvidia-exclusive long-term. GPUs as far back as the GTX 900 series from Nvidia have HEVC encoding, and the vast majority GPUs going back far longer have H264 encoding — so nearly anyone can use this, eventually.

Have slow internet and don’t see yourself being able to include more tracks on top of what you normally send? That’s fine! You can still enable Enhanced Broadcasting and the service will still optimize your settings best for your system and internet connection — and once the newer codecs are available, you’ll end up being able to send a lot more than you’d expect, anyway.

Twitch should begin inviting users to the beta this week, but be warned: Invite groups will be small. Be prepared to wait, but know that if you participate you are truly helping shape the future of Twitch. The ultimate goal here is improving the viewer experience on Twitch, and that is what really matters, isn’t it?

Author: Adam Taylor, Contributor