Top 5 Methods of Low Latency Live Streaming

We live in an always-accelerating world of increased interactivity and connectivity.  With that comes the rise of live video streaming applications where delays of even one second ruin the enjoyment of events or conversations. Low latency live streaming is necessary for preserving the integrity of the viewer’s experience.

As customers, clients, and users demand better live experiences, a video delay measured in seconds will never be good enough. In a high-speed lifestyle full of immediate, spoiler-inducing text messages and notifications, only sub-second latency can actually create real-time experiences.

The race to zero latency is well underway, so let’s take a look at the current state of low latency live streaming protocols and see which of these top five technologies actually achieve sub-second real-time latency.

#5 WebSockets and Media Source Extensions (MSE)

WebSockets and Media Source Extensions (MSE) can be used together to deliver live streams. This is the approach used by NanoCosmos and Wowza’s proprietary WOWZ technology to achieve a latency of between one to three seconds.

This is a very creative approach to solving the latency problems in live video, and NanoCosmos should be commended for being one of the first groups to come up with it. Despite this intelligent use of JavaScript, there are still drawbacks to this approach.

Browser implementations of the MSE API are too slow to process the video within an acceptable time and thus it introduces significant latency. Additionally, MSE is currently not supported on iOS devices. WebSockets are also limited to using TCP which can cause packet back-up in certain network conditions. Lastly, WOWZ currently does not support Adaptive Bitrate which will create a negative user experience in less than ideal (i.e. real-world) network conditions.

Pros:

• It doesn’t rely on HTTP making it an interesting and somewhat clever approach.
• It’s simpler to implement than a WebRTC approach.

Cons:

• Not a Web Standard.
• MSE is currently not supported by iOS.
• As a non-HTTP based solution, it cannot be supported by CDNs.
• TCP based and misses out on the advantages of UDP.

Biggest Con:

• No real-time latency.

#4 Apple’s Low Latency HLS – ALHSL

Apple extended the HLS protocol creating their own low latency HLS. It generates parts; partial segments of TS or CMAF chunks. Those parts are advertised at the head of the HLS playlist. Using HTTP 2 push, players can download smaller groups of frames quickly after they are encoded. However, this still results in latency from 3 – 7 seconds.

Not only is this too high, but additional configurations on CDNs are needed for widespread support.

Pros:

• HTTP Push means that you are no longer polling for the stream.
• Playlist deltas offer a huge advantage of a single playlist for long-running streams.

Cons:

• HTTP Push is not currently supported on CDNs.
• Not a community-based standard (controlled by one company).
• Extra complex without real-time latency results.

Biggest Con:

• No real-time latency.

#3 CMAF

Created in 2017 through a joint effort between Microsoft and Apple, CMAF is a standardized container that can hold video, audio, or text data which is deployed using HLS or DASH. It’s worth noting that Apple’s LHLS also leverages CMAF, and what we are describing in this section is plain vanilla CMAF used with either HLS or DASH. The advantage of CMAF is that media segments can be referenced simultaneously by HLS playlists and DASH manifests. This allows for the support of various devices without having to store multiple copies of the same stream content with different encoding for different formats.

However, this attempt to achieve low latency, requires many small chunks of 250 milliseconds. This creates higher bandwidth video, adds to the number of HTTP calls happening (at least four per second), and increases server load. Accordingly, current proof-of-concepts when deployed over the open internet show a sustainable Quality of Experience (QoE) only when the end-to-end latency is around 3 seconds at best.

Cons:

• Breaks current bandwidth detection schemes.
• Creates higher bandwidth video. only achieves around 3 seconds of latency.

Biggest Con:

• Three seconds of latency: not real-time.

#2 Community Low Latency HLS – LHLS

Community-created low latency HLS (LHLS) is a joint effort of the HLS.js community with others including Mux, Twitter, JWPlayer, Wowza, Elemental and Akamai to collaborate on a community-driven approach to implement low latency streaming using HLS.

LHLS is able to reduce the latency by leveraging HTTP/1.1 chunked transport for segments and announcing segments before they are available thus allowing a player to request the segments as they are produced. This approach is very similar to CMAF with the main difference being that LHLS uses the MPEG Transport Stream container. Overall, this produces about 3-5 seconds of latency. The community version of LHLS has one huge advantage over Apple’s protocol in that it doesn’t require HTTP.Push.

Pros:

• Created by many different companies and could become a web standard due to the cooperative nature of the community.
• Can leverage existing HTTP based CDN infrastructure with no changes.

Cons:

• Doesn’t solve for long playlists on long-running streams.
• Creates many chunks which can be too much overhead for a CDN.
• Still uses HTTP for fetching chunks which is a lot over overhead versus a push-based protocol.

Biggest Con:

• Three seconds of latency (at best).

#1 WebRTC

WebRTC (Web Real-Time Communication) is a standards-based, open-source project supported by Google, Microsoft, Mozilla, Opera, and Apple. It was designed and built around reducing latency to the absolute minimum. It also works without requiring additional plugins or downloading native apps.

WebRTC was originally designed for creating video calling applications in the browser, and thus requires some creativity and a complete rethink of traditional CDN architecture to achieve one to many scale. It’s also a complex protocol with many layers requiring specialized knowledge.

As opposed to the other approaches outlined above: WebRTC is UDP based rather than TCP based. Unlike TCP, UDP is not concerned with the order of the data, rather it delivers each packet to the application the moment it arrives.

Cons:

• Complex and difficult to implement (luckily we did the hard work for you).
• Disrupts the traditional HTTP based approach used by CDNs.

Pros:

• Fully scalable on cloud infrastructure.
• Widely supported Web Standard.

Biggest Pro:

• The only currently viable solution that produces REAL-TIME LATENCY at scale.

Red5 Pro and WebRTC

This is why we at Red5 Pro have integrated WebRTC into our solution to unlock sub 500 milliseconds of latency. We believe that the future of low latency live streaming is fully interactive, and the kinds of “low latencies” that users put up with today will be intolerable within the next five years. Any protocol that still measures latency in seconds is– at best– a stopgap solution in a race to zero latency.

For those looking for more details please take a look at our whitepaper. For any questions, comments, or a lively debate on the many merits of various live streaming protocols, send a message to info@red5.net or schedule a call.