An intriguing set of new capabilities for bringing IP content to set-top boxes is coming into focus as a result of the inclusion of the GStreamer multimedia framework in the protocol stack endorsed by developers of the Reference Design Kit platform for next-gen devices.
Barcelona-based Fluendo, a leading purveyor of advanced solutions utilizing GStreamer, earlier this year opened North American headquarters in New York City with the goal of expanding its base in all the market sectors it serves, including the set-top segment, says Fluendo CEO Muriel Moscardini. “Our opportunity in the U.S. is to equip companies with a multi-platform solution that saves time to market and cuts costs,” Moscardini says. “RDK has greatly added to the appeal of our solution for the set-top market.”
Over the past ten years Fluendo, a key contributor to development of the open source GStreamer framework, has worked with OEMs and chipmakers worldwide as a supplier of fully licensed codecs integrated into player applications for multiple sectors, including enterprise multimedia, video production, home entertainment, thin-client computing, OTT and smart TVs as well as set-tops. Named customers include NAGRA, Technicolor, Sony, Toshiba, HP, Samsung, IBM and many other companies.
Where RDK is concerned, Moscardini notes that Fluendo’s GStreamer-based Oneplay platform opens the door to building rich applications on a framework that supports playback in Linux, Windows, MAC OS X, Android and iOS environments from video streamed over any of three leading adaptive bitrate (ABR) formats, including HLS (HTTP Live Streaming), Microsoft Smooth and MPEG DASH adaptive bitrate formats. “Ours is a middleware position where we’re behind applications people build to help service providers differentiate their services,” she says.
As explained by RDK Management LLC on its website, GStreamer has been included as an “open source software to provide a fully-tested, stable and powerful way to enable RDK developers to deliver world-class multimedia streaming applications. RDK uses GStreamer extensively to stream media securely within the home network with a full set of components for managing complex media across networked devices.”
RDK’s selection of GStreamer attests to the maturation of the framework over many years of development by contributors who set themselves the arduous task of making it possible for developers to build players that would support multiple codecs, filters, ABR muxers and demuxers, DRMs and other essential components to enabling rich multimedia experiences, initially for Linux OS environments. Two years ago, Fluendo collaborated with the U.K. consultancy Collabora to create the GStreamer SDK (software development kit), a free resource available at GStreamer that broadened the availability for developing full-featured multimedia applications across all the leading desktop platforms.
GStreamer accomplishes its magic using a plugin model where hundreds of software modules in the database accessed by developers are configured with APIs that allow them to be slotted into the framework with the click of a mouse. These plugins, positioned in the library under different categories such as codecs, filters, sources, protocols, etc., can be mixed and matched across a set of arbitrary developer-defined “pipelines” that steer the flow of data passing through the selected plugins to support a full-fledged multimedia application.
Capitalizing on the capabilities brought together with ongoing advances in GStreamer and the introduction of the GStreamer SDK, Fluendo earlier this year was able to consolidate its platform into the single Oneplay development platform, where the Oneplay engine can be used to enable GStreamer-powered media playback on applications for all supported platforms. “We created Oneplay as a brand to represent the fact that you can build applications on our platform to play anything,” Moscardini says. “What we have behind the brand is support for development of different solutions adapted to specific customer needs.”
One of Fluendo’s latest steps is the inclusion of an H.265 High Efficiency Video Coding (HEVC) codec developed by Berlin-based Fraunhofer Heinrich Hertz Institute, a leading contributor to HEVC, as part of the Oneplay codec portfolio. “By adding Fraunhofer HHI’s decoder to Fluendo’s proven multimedia player, Oneplay becomes one of the few media players in the world that is able to support HEVC,” Moscardini says. “This gives us an opportunity to work with OEMs to include H.265 now, which is an important attribute for many device manufacturers at this point, even though there’s very little 4K content available.”
As with other codecs, a big benefit Fluendo brings to customers is it has taken care of all the licensing requirements for use of H.265, eliminating that headache from their workloads, she adds. “The main vision of Fluendo from the beginning was to create a one-stop shop where customers who come through us obtain the rights to distribute using H.264, MPEG2, MP3, Dolby, etc.,” she notes. “This was a major factor in our ability to build a customer base with major companies.”
Of course, in the set-top environment as presently constituted software-based decoding is not a factor, but with miniaturization of devices it’s not out of the realm of possibility. But, more likely, it’s the other capabilities embodied in the Oneplay portfolio that will hold appeal for OEMs developing RDK-compliant products.
When it comes to utilizing hardware resources, another part of the Fluendo playbook has been to partner with chipmakers like Intel, STMicroelectronics, AMD and others to make sure that player applications running on their chipsets are able to leverage processing power to maximum effect. “You never know what processing environment your player will be running on or whether it will be able to use acceleration to enhance performance,” she says. “Our work with SoC manufacturers has been a big help to OEMs.”
For example, Fluendo is a member of the Intel Consumer Electronics Network, a community of hardware, software and services providers that contributes to accelerating the development, time-to-market and scalability of Internet-connected CE devices running on Intel chipsets. Fluendo has collaborated with Intel to enhance GStreamer support for Intel’s media processor CE 3100 and the Atom CE4100, which are used in set-top boxes. Intel now provides a plugin to the GStreamer framework that accelerates video decode and rendering by processing and offloading the data to the Atom processor E6xx series video engine through the Intel EMGD driver, according to Intel documents.
RDK, of course, has made sure to include alternatives to a player-based approach to enabling IP services on next-gen set-tops and gateways. As Ken Morse, CTO for connected devices at Cisco Systems notes, developers have also taken into account the impact on middleware design taking shape with maturation of HTML5 technology.
“RDK is basically a community-sourced environment that allows us in conjunction with our customers to more rapidly innovate and bring product to market in a faster time,” Morse says. “When you look at RDK, it’s built on top of varying hardware environments, whether it’s IP set-tops, gateways or traditional video QAM set-tops. And then you use standard open-source components. Whether it’s things like GStreamer to enable the playback of video and audio, whether it’s things like QT Webkit for enabling rich HDML, our customers will have the opportunity to create rich user experiences in conjunction with moving content around both within the device and within the home to whatever devices are there.”
As Moscardini acknowledges, in the OTT world at large player plugin technology is now in intense competition with surging reliance on HTML5 as a way to enable execution of multimedia experiences across multiple types of devices directly from websites in HTML5-compatible browsers. “A lot of TV apps are now built out of browsers using HTML5,” she says. Of course, she adds, browsers powered by GStreamer can be used to similar effect, as is the case with the use of GStreamer by Firefox in Linux-based OS environments.
But the question for MVPDs is whether they can rely on HTML5 and HTML5-optimized middleware to accomplish the holistically integrated TV-quality of experience required for delivering IP streams to home gateways, set-tops and other devices. As evidenced by the decision of Netflix last year to eventually replace its Silverlight plugin approach with HTML5-based streaming, the OTT community has wholeheartedly embraced the universal app approach made possible by HTML5.
But, notwithstanding the W3C’s (World Wide Web Consortium’s) development of Media Source Extensions (MSEs) that enhance HTMLMediaElement specifications to allow JavaScript to generate media streams for playback, HTML5 is still an incremental work in progress tied to the pace of browser ecosystem adoption to the standard. Specifically, the ABR streaming and DRM protection support enabled by those extensions are supported by very few browsers, even though the baseline HTML5 is now supported in virtually all recent versions of the leading browsers.
Netflix now supports HTML5 streaming to Internet Explorer 11 in Windows 8.1, Google Chrome OS (but with Chrome-specific plugins to cover certain gaps in Chrome’s adoption of all the MSEs) and Safari OS X Yosemite,which launches this fall. But in a recent blog Anthony Park, vice president of engineering at Netflix, and Mark Watson, the firm’sdirector of streaming standards acknowledge there’s a long way to go, saying they “look forward to a time when these APIs are available on all browsers.”
All of which points to the need for pay TV-optimized solutions when it comes to using HTML5 with RDK. As previously reported, this is precisely what Espial, a pioneer in HTML5-based app development, has done by using HTML5 with proprietary enhancements suited to maximizing performance of the company’s cloud-based middleware on set-top boxes.
In other words, when it comes to the managed MVPD service environment, HTML5 is a tool requiring significant enhancements in proprietary middleware executions. It will be up to purveyors of the solutions tied to the GStreamer player framework on the one hand and cloud-based HTML5 middleware on the other to convince potential customers theirs is the best approach. Fluendo’s input on the question will make for interesting internal debates among decision makers in the months ahead.
