With the volume of Internet video traffic soaring ever higher, the prospects for a surge in per-stream bitrates triggered by new video formats is adding new urgency to the perennial search for ways to keep bandwidth consumption under control.
Even though HEVC (High Efficiency Video Coding) has gained a significant commercial presence as the MPEG-ordained successor to AVC (Advanced Video Coding), the market is awash in new codecs and other solutions which backers claim can get the premium content industry closer to where it needs to go as 4K UHD, HDR, virtual reality and possibly even a glasses-free version of 3D enter the video flood tide. It’s a two-front battle, waged on the one hand by codec suppliers offering what they say are better performing and, in some cases, lower cost alternatives to HEVC and, on the other, by vendors and distributors with solutions they say will dramatically cut bandwidth consumption no matter which codec is in use.
Given what’s in store as higher quality, multi-dimensional video streams fill the pipelines, industry players may well end up choosing solutions based on both approaches. 4K UHD requires transmission of four times as much data as a 1080p HD signal, translating to an uncompressed bitrate of 12 gigabits-per-second vs. 3 Gbps. HDR (high dynamic range) adds what has been variously estimated as anywhere from 20 to 40 percent to the throughput, whatever level of resolution is in play.
Virtual reality (VR) will consume anywhere from 150-200 Mbps per stream using AVC compression, according to CableLabs. The 3D quotient, if it materializes now that glassless 3D technology is going into commercial use, will depend on the type of technique employed but will be significant.
So far, the consensus among MVPDs and major OTT distributors is that H.265, cutting the bitrate by up to 50 percent from what’s required with H.264, is the solution to handling all these requirements. But the chaotic outpouring of new compression technologies lined up to compete with HEVC, also known as H.265, suggests that consensus could fall apart as the bandwidth pressures increase.
As Thierry Fautier, vice president of video strategy at Harmonic, notes in a recent blog, “[W]e might see a lot of new tools that will not only improve compression efficiency, but will also enable highly scalable cloud-based encoding. In addition, as we see more software-based decoders on the market, we can also expect more flexible schemes for codec upgrades (e.g., not having to wait 10 years to get a new codec).”
And, of course, software solutions designed to work on commodity servers and end user devices without requiring purpose-built microprocessors to handle the processing load have the potential to upend the traditional lockstep conformity to a dominant standard. With practical, high-performance software solutions that can work on general-purpose CPUs used with handhelds and smart TVs as well as PCs, distributors will have the flexibility to choose and propagate with dedicated client player software whatever solution works best for them.
So far, says David Mowrey, vice president for product management at Clearleap, which was recently acquired by IBM, nothing has emerged to challenge the efficiency of being able to deliver compressed content across an ecosystem where implementation of a standardized codec provides assurance the content will be viewable by all users on all devices. Dealing with advanced software-based codecs that work in some device environments and not in others is a headache, Mowrey adds.
“It’s technically feasible,” he notes. “But dealing with multiple players and doing things on different platforms incurs management and maintenance costs, including the costs of acquiring licenses for different codecs running on different players.” While it would be nice to have a next-generation software codec that would work across all device platforms, “I haven’t seen anything like that,” he says.
But it’s likely he will before long. The list of developments in compression technology that bear watching is a long one, including:
V-Nova’s Perseus
As previously reported, U.K.-based V-Nova claims dramatic gains over HEVC for Perseus, a software codec designed to work on existing server and device platforms that has gained substantial backing with a coterie of 36 partners who have integrated Perseus into their solutions. Named partners include Broadcom, Broadpeak, Dell, Elber, Encompass, the European Broadcasting Union, Hitachi Data Systems, Imagine Communications, Intel, Nokia, NTT Data<, Nvidia, Sky, ST Microelectronics, Thomson Video Networks,VideoFlow, VisualOn and Wyplay.
“We’re a software company playing into the existing infrastructure to turbocharge its performance,” says Fabio Murra, senior vice president of product and marketing at V-Nova. “Since we introduced Perseus at NAB (2015), we’ve been engaging suppliers as partners who provide support in both the contribution and distribution markets.”
Utilizing less processing power than HEVC, Perseus can encode 4K UHD content for streaming over live or on-demand service feeds at just 7-8 Mbps, or about a third the bitrate required for transmission at similar quality over HEVC, Murra says. One of the first commercial applications for distribution to consumers is in play with NTT Data, the Japanese telecom’s global IT service provider, which has introduced Perseus with on-demand and live encoding solutions used by Sky Italia and other customers.
In B2B contribution applications running on CPUs or GPUs, Perseus achieves lossless compression of 4K UHD at 300 Mbps, enabling three feeds rather than one over a 1 Gbps pipe, Murra adds. This was recently validated in quality tests performed by Germany’s Institut für Rundfunktechnik.
“We process things very efficiently,” Murra notes. “Rather than taking the macro block approach commonly used by other solutions, we perform processing hierarchically in a massively parallel manner.”
This allows the components in multi-core silicon to work simultaneously across the entire image, thereby identifying all correlations and differences between them much more efficiently, he explains. For example, Perseus can achieve the touted bitrate reductions encoding 4K UHD at 60 frames per second in real time on two standard eight-core Intel Xeon processors, which are running on servers that populate datacenters worldwide. This is well below the 32-core density of the latest Xeon processors.
As an alternative scenario, V-Nova’s partnership with GPU supplier Nvidia has made it possible to run Perseus on a GPU chipset without additional hardware to perform real-time encoding well below the 16 milliseconds necessary to process a full frame 4K UHD picture at 60 fps. Murra says Perseus GPU-powered products now available from equipment vendors and system integrators are being deployed in video contribution applications, delivering high-quality feeds from events and through the backbones of Tier-1 operators and service providers.
Adding to these eye-popping performance capabilities, Perseus at the decoding end is designed to default to legacy MPEG codecs in instances where devices aren’t equipped with processors sufficient to the full capabilities of the V-Nova codec. For example, if a device can only decode MPEG-2, that’s what it will do on the Perseus-encoded asset, which means there’s no need to perform multiple encodes to reach multiple device types.
Google’s VP9/10
Google has been in an uphill battle for several years seeking to woo the market to what it bills as royalty-free technology offered through the VP series of codecs, which it started promoting after its acquisition of the developer of the technology On2 in 2010. The latest iteration, VP9, touts capabilities comparable to HEVC, making it ostensibly an obvious choice for anyone wanting to avoid licensing costs imposed by MPEG licensing pools.
But the royalty-free incentive hasn’t overcome market resistance to moving from a world standard to a proprietary system. Instead, VP9, which requires processing power comparable to that of HEVC, has gained a toehold by virtue of the fact that Google uses the codec for all 4K content delivered over YouTube and has embedded it in the Chrome Web browser for use with 4K content.
VP9, with support from chipmakers Broadcom, MediaTek, Nvidia, ST Microsystems and Sigma Systems, is running on Android devices like the Samsung Galaxy S6, game consoles from Sony, LG and Sharp and in most smart TVs entering the market over the past year. But with holdouts including all Apple devices and Microsoft PCs and consoles, Google hasn’t achieved anything approaching universal support.
The company hopes to do better with release of its next codec, VP10, which it says will be in the market by the end of 2016 or early 2017 to offer a 50 percent improvement in performance over H.265 and VP9, which would cut the bitrate on 4K streaming to about 10 Mbps. The release will be timed to catch the wave of a new generation of device chipsets that can handle the greater processing requirements, estimated to be about 40 percent more than is required with VP9 and H.265.
The Alliance for Open Media
Paralleling its efforts with VP10 and potentially offering a greater threat to HEVC and succeeding MPEG standards, Google in September 2015 joined with Amazon, Cisco Systems, Intel, Microsoft, Mozilla and Netflix to develop a new generation of royalty-free codecs through the Alliance for Open Media (AOM). This is a significant amassing of market power missing from Google’s standalone battle against the consensus standard, notably bringing into the fold two steadfast opponents of the VP codecs, Microsoft and Mozilla.
AOM pools the technical knowhow behind several independent codec initiatives. Along with Google, IP contributors to the effort include Microsoft, with its long history of work on the VC-1 codec, now in use as a standardized alternative to AVC; Cisco, which developed Thor as a royalty-free codec for use with its videoconferencing products; Mozilla, which developed Daala as a royalty-free option to be used with Foxfire in lieu of support for AVC or HEVC, and Intel, a long-time developer of compression technology.
But while AOM members have confirmed they intend to cooperate in pooling IP for a new codec, they’ve said very little about performance targets or how they will go about making the royalty-free technology an industry standard. Cisco and Mozilla are participants in another royalty-free next-gen codec initiative underway through the auspices of the Internet Engineering Task Force known as NetVC, a still vaguely defined project that has set sometime in 2017 as the goal for initial release.
Whether or not work within AOM flows into or becomes the anchoring force behind NetVC remains to be seen. But whatever emerges from these efforts will, like VP10, be dependent on a new generation of microprocessors with potential limitations tied to the relatively slow spread of such processors into the ecosystem.
RealNetworks’ RealMedia HD
RealNetworks in recent months has announced software and hardware versions of a new codec that was introduced into the Chinese market last year, where RN has been a major supplier of codecs for video distribution. Developed by its sizeable engineering staff in Beijing, RealMedia HD (RMHD) is a work in progress with compression ratios presently equaling those of HEVC, according to company announcements and other documents.
RN is claiming significant cost savings will be realized through lower CPU usage and battery consumption with RMHD compared to H.265 but hasn’t provided specifics. The company says the software version, which entered the Chinese market in July 2015, has been integrated into streaming vendors’ and OEMs’ media players in that country, ensuring a smooth transition to HD, 4K and 8K as distributors move beyond SD content that utilizes the widely deployed RealMedia Variable Bitrate (RMVB) technology.
As a result, the company says, viewers will experience strong video quality with low bandwidth requirements, while chipset manufacturers will see large cost savings in comparison to their current hardware. OEMS will also benefit from the backwards compatibility that supports older RealMedia content. A major advantage for chipmakers utilizing the hardware-optimized version is the fact that new gains in performance, which the company says are in the development stage, can be easily implemented on chipsets that support the RMHD architecture, avoiding the need for new chip making cycles. At the same time, RN says, RMHD is designed to evolve with advances in manufacturers’ chip designs so that it can be readily scaled with new chips entering the market.
While there are royalty costs, RN says it has a straight-forward approach that overcomes the confusion associated with licensing in the HEVC domain. But, with so little known at this point, it’s hard to gauge the RMHD potential in the global next-gen codec competition. Clearly, though, with a substantial base of adoption in China to work from, the company has market momentum behind its ongoing development efforts, which makes it likely the performance levels will at least keep pace with competing technologies in the years ahead.
Tveon
Least known among the new generation of codecs vying for international adoption is the solution developed by Canada-based Tveon Systems. A new player in the global HD and UHD compression market, Tveon claims its new codec can stream 4K at 2 Mbps and 1080p HD at 200 Kbps, the most aggressive claim yet made for any codec in the market.
The company says the technology enables “real-time compression for distribution of live broadcast and/or VOD for both HD and UHD content,” but provides no detail on how it works or whether it can be implemented in software on current generation CPUs. We only include mention of the technology here as a heads-up insofar as Tveon has not provided independent validation of its claims and has no commercial deployments to point to other than the fact that a Canadian systems integrator and project manager, FourNetworks, whose clients include Telus and AT&T, has taken a 30 percent stake in the company.
The MPEG Trajectory
As supporters of HEVC note, the platform is comprised of an evolving set of profiles, which, like past MPEG codec profile sets, will continue to improve, leading by various estimates to another 25-50 percent gain in performance before the platform runs its course. And beyond HEVC the legion of companies participating in MPEG profile development are participating in MPEG’s FutureVideoCoding initiative, which aims to deliver a superior new codec by 2020.
Given that technology trajectory, perhaps the biggest impediments to HEVC dominance over competing technologies in the years ahead will be the costs and complications of licensing the technology. Last year some leading contributors to the HEVC IP pool broke away from the fee structure set up by the MPEG LA licensing authority to form a separate licensing mechanism, HEVC Advance, to counter what participants considered to be unacceptably low terms set by MPEG LA.
While HEVC Advance at year’s end revised its terms in response to strong market blowback, the cost structure is still more draconian than the one set by MPEG LA. Moreover, anyone who wants to deploy HEVC will be burdened with determining where to get licenses based on which components of the overall profile set they want to use.
Adding to the confusion, Technicolor, a major holder of HEVC IP and a founding member of HEVC Advance, has now pulled out of both pools to offer more lenient terms through its own licensing process. Technicolor is urging other vendors to do likewise, apparently out of concern that the current royalty fee structures could squelch HEVC adoption as royalty-free competitors gain traction.
MPEG has a royalty-free initiative underway with development of what it calls IVC (Internet Video Codec). That platform, now in the committee draft stage, is expected to move to standardization later this year, but as a no-cost solution targeted to run-of-the-mill OTT video needs, it will have little bearing on market decisions driven by high-end video requirements.
In any event, whatever flows from MPEG’s FutureVideoCoding initiative, the stability of the MPEG consensus depends on how HEVC fairs. While current generation CPUs in PCs, Macs and many tablet models have sufficient power with built-in high-end GPU components to decode HEVC as well as other compression formats included with software players, there’s a bigger hurdle for handsets, smart TVs, set-tops and other devices that use lower cost processors.
The biggest headway so far can be seen on the mobile front. With the generational turnover in handset models over the past two years, leading device manufacturers have been implementing chipsets optimized for HEVC, with the result that there are now 313 million HEVC-capable handsets in consumers’ hands worldwide, according to research conducted by Digital Tech Consulting. DTC projects that base will reach 1.5 billion by 2020.
“The mobile industry moves a lot more smartphones than the consumer electronics industry moves TVs,” notes DTC analyst Jing Sui in a recent blog. “To date, Apple’s iPhones 6/6S and iPhone 6/6S Plus natively support HEVC for Facetime; Google’s operating systems, Android 5.0 and above, include native HEVC support; and mobile chip makers such as Qualcomm, MediaTek, Intel and Samsung all include HEVC support in their latest products.”
Whereas 4K UHD content is the main driver for HEVC adoption in the pay TV market, the emergence of video as a major application on mobile networks has created a more urgent demand for improved compression ratios. “If the mobile industry wants to be the next primary video provider to consumers, they must solve their infrastructure capacity problem,” Sui says.
Smart TV manufacturers, too, are now including HEVC support in their chipsets, and, while the slow pace of 4K UHD rollout in premium video services, especially those offered over legacy MVPD networks, has gated implementation of HEVC-capable set-tops, such models are widely available for deployment when the market is ready. ABI Research predicts annual global shipments of HEVC-capable set-tops will reach 88 million by 2020.
The bottom line is that, in the grand scheme of things, these penetration numbers point to a long period of adoption before HEVC can become the dominant next-gen codec. In a new report on the HEVC encoder market Frost & Sullivan says it likely will take close to eight years “for HEVC to play a dominant role in the video ecosystem.”
Over that time a market shift to HEVC that’s based on past approaches to adopting the next generation of MPEG technology may not pan out as anticipated. As Greg Rutz, a lead architect in the Advanced Technology Group at CableLabs, notes in a recent blog, the case for a royalty-free solution is improving with the emergence of new options and confirmation that VP9 measures up technically to HEVC.
“From a performance standpoint, we feel that VP8/AVC and VP9/HEVC are very similar,” Rutz says. “Numerous experiments have been run to try to assess the relative performance of the codecs and, while it is difficult to do an apples-to-apples comparison, they seem very close.”
Looking at the range of royalty-free options in play, he adds, “Not sure we can say this statement with utmost confidence; but it seems clear that the confusing landscape of IP-encumbered video codecs is driving industry towards a future where digital video…may truly be free from patent royalties.”
In this new era of incessant gains in chipset performance, generic rather than purpose-built processors even on the smallest video-capable devices will be able to support just about any software codec running in players downloaded to those devices. As a result, the use of compression technologies as more advanced video formats come into play could become less uniform across an ecosystem where individual choices are based on performance and costs.
Beyond the Codec Wars
Meanwhile, providers of ways to improve performance with use of already deployed codecs, including AVC as well as HEVC, are making significant progress bringing their solutions to market. For example, as previously reported, MediaMelon’s QBR (Quality Bit Rate) technology, working in live and on-demand streaming environments, uses intelligent analysis in the adaptive bitrate (ABR) streaming process to identify which of the multiple bitrates encoded for each segment of a video stream is best suited to delivering that segment at the desired quality level, resulting in a per-session average bitrate reduction of about 35 percent.
Given the solid foundation now in place with AVC and other codecs, the market is ready to look for ways to improve on bandwidth efficiency that can obviate moving to a new generation of codecs, suggests MediaMelon CEO Kumar Subramanian. “With the commoditization of media players people are looking for tools that can improve on video quality and bandwidth usage within the existing infrastructure,” Subramanian says.
He notes MediaMelon is working with a wide array of content owners, distributors and vendor partners to promote adoption of its QBR video processing platform and its player, which is available for use with iOS, Android and popular web video players such as DASHjs and Exoplayer and can be easily integrated with smart TVs, custom player and CE devices. Announced partners include Deluxe OnDemand, video encoder vendor Media Excel and
NexStreaming, which supplies the multiscreen HLS player SDK known as NexPlayer.
Beamr is another provider of bandwidth-saving mechanisms that run ancillary to AVC and HEVC encoders, in this case applying perceptual image quality analytics to optimize bitrates. As confirmed in testing by M-GO, an OTT distributor that is among the first to use the technology, the Beamr Video optimizer cuts bitrates by up to 50 percent with no sacrifice in quality.
M-GO has also validated that its use of the Beamr technology consistently cuts the frequency of rebuffer events on client devices by up to 50 percent, decreases start times by up to 20 percent and makes higher quality experiences available over low bitrate access links than would otherwise be possible, according to M-GO CTO Samir Ahmed. “We found Beamr’s technology to deliver network-friendly streams with excellent image quality, resulting in enhanced user experience and significant cost savings,” Ahmed says.
With these capabilities in place, M-GO has been able to take an aggressive role with introduction of new content formats, including 4K UHD, 4K with HDR enhancements and virtual reality. Users can stream to rent about 60 4K titles at the present time, with more being added as they become available from the studios. At this point 4K and HDR content is only available for viewing on Samsung 4K TV sets.
Beamr Video, currently used with encoding of on-demand content, will be available for linear programming this year, says Beamr president Eli Lubitch. In addition, he notes, “We’ve made Beamr available in the AWS (Amazon Web Services) cloud environment, where customers can implement the technology on AWS servers using our turnkey service or by doing the integration themselves.”
There are other approaches to improving bitrate performance on existing codecs under development across the ecosystem. Netflix, which has taken an aggressive role with introduction of 4K-formatted content, recently made headlines with word that it was employing in-house developed video processing techniques that allow it to cut bitrates by 20 percent with no loss in quality.
The key, Netflix says, isn’t use of a new type of codec but rather algorithmic processing performed on files entering the H.264 (AVC) or H.265 (HEVC) encoding process to determine what the optimal bitrate profiles or “bitrate ladder” should be across the length of the content as parameters vary from frame to frame. In other words, whereas the built-in encoding algorithms take into account such parameters to determine how much information in a given frame is new and therefor needs new processing treatment, Netflix is taking into account more detail specific to each file, including the optimal resolution for each sequence within the file, the end user’s bandwidth limitations and the screen resolution of the receiving device to determine the optimal bitrate for a given level of quality.
This “per-title encoding” process exploits the flexibility of Internet streaming to vary bitrates with each fragment or “chunk” a client pulls from the server, says Anne Aaron, manager for video algorithms at Netflix, who with several colleagues has authored a blog explaining their approach. “To deliver the best quality video to our members, each title should receive a unique bitrate ladder, tailored to its specific complexity characteristics,” Aaron says. “To design the optimal per-title bitrate ladder, we select the total number of quality levels and the bitrate-resolution pair for each quality level according to several practical constraints.”
The process takes into account things like the fact that, at a given level of resolution for a given sequence, there’s a top bitrate threshold beyond which quality doesn’t improve with higher bitrates, meaning that the system caps the bitrate for that sequence at whatever that threshold is to avoid consuming unnecessary bandwidth. Or, in instances where encoding a sequence at a higher resolution reduces the quality at a given bitrate, which might represent all the bandwidth available to the user for that sequence, the system chooses the lower resolution to keep quality as high as possible for that bitrate.
All these developments ancillary to codec technology could have a profound impact on how the market reacts to the lineup of next-gen codecs. Once someone has gone to the trouble to develop algorithms that can automate application of these solutions with minimal disturbance to existing infrastructure, the benefits to systems already in use can put the distributor a generation ahead on the compression development curve and keep them there as they adopt new compression modes.
But, sooner or later, even with use of these techniques, distributors will have to deal with the unrelenting increase in video flows by introducing new codecs. For anyone hoping to deliver a competitive consumer experience, it’s clearly advisable to avoid making choices based on complacent adherence to conventional wisdom.
