Suddenly, it seems, anyone moving into the advanced advertising and apps spaces who don’t have an automatic content recognition (ACR) strategy in play risks being left behind in the race to facilitate efficient use of connected devices to drive new revenues.
ACR applies to a range of techniques that can employ audio and/or video fingerprinting, digital watermarking, or, in at least one case, “video signature” to trigger a real-time implementation of an ad or app in conjunction with what any given user is viewing or listening to. “The beauty of this technology is we can use existing devices without huge infrastructure issues,” says Alex Terpstra, CEO of Civolution, which is one of many companies bringing ACR capabilities to market.
Civolution previewed its forthcoming ACR solution, which can be used with either the firm’s fingerprinting or watermarking technologies, at the recent NAB Show in Las Vegas. “We port our software to devices like tablets and smartphones where we can help applications developers synchronize the applications automatically to the main [TV] screen,” Terpstra says.
“If you would have a TV show, even a live show, that has an appealing app on a tablet device, a lot of things could happen in that app that is related to the actual program on TV if you were able to accurately synchronize the two,” he says. “That’s what we do with our watermarking and fingerprinting technologies.”
ACR has been in play for a while in a wide range of applications, such as the Nielsen Audio Video Encoding (NAVE) system, the audio watermarking element of which for the past decade or so has been used as a way to identify and track program content for TV ratings purposes. The Nielsen meter equipped with NAVE capabilities registers what the viewer is watching automatically by identifying the source of the audio watermark in the program.
Another long-standing ACR user is Shazam, the provider of fingerprinting-based music discovery apps for mobile devices, which last year extended its business with the introduction of the Shazam Audio Recognition Advertising (SARA) program. First used by clothing brand Dockers in the “Wear the Pants’ campaign during the 2010 Super Bowl, SARA employs Shazam’s music recognition system to let brand advertisers tag a commercial for direct interaction with the Shazam user base, now numbering over 100 million people worldwide.
The tag induces viewers to engage with the promotion by pointing their mobile devices in the direction of the commercial and hitting the Shazam button on the device. Shazam’s audio technology recognizes the specific advertisement and returns a customized result to the mobile device.
In a sign of how hot ACR has become, Yahoo! last month acquired a three-month-old startup in the field, IntoNow, for a reported $20-$30 million, winning a battle to buy the firm that included Twitter and Facebook, according to the online outlet TechCrunch. IntoNow, led by former Google and Viacom executive Adam Cahan, uses audio fingerprinting to allow users to identify and share television programs with their friends via iPhones.
Yahoo! believes the technology will help socialize the video programming offered through its platform. “Relying on social channels as a means for discovering content – whether it’s on a PC, mobile device, or TV – is rapidly on the rise,” says Bill Shaughnessy, Yahoo!’s senior vice president of product management. With a growing database currently comprising 150 million minutes of content, IntoNow can fingerprint TV shows in real-time with 99 percent accuracy, without requiring the participation of the networks.
But there’s a big advertising play as well, as revealed by IntoNow’s recent deal with Pepsi, which allows users to respond to commercials in a new campaign for Pepsi Max. The first 50,000 viewers who use IntoNow to tag a Major League Baseball-themed Pepsi MAX commercial will receive coupons for a free 20-ounce soda at retailers like Target and CVS. The company says automotive and movie advertisers are likely to be the next entities to sign on to the platform.
While the applications are similar, there’s a big difference between how watermarking and fingerprinting technologies get the job done. In the case of audio or video watermarking, the identification of the content is based on a digital code embedded invisibly in the audio or video stream that can be readily matched against a database containing all such codes. Fingerprinting, on the other hand, relies on matching marked slices of content with the same slices in a database, which entails a search across the database, typically in a two-step approach that first finds a set of likely matches and then drills down to get to a precise match.
Both technologies have their drawbacks and advantages. Watermarking is subject to precise placement of the identifying code where it won’t interfere with the content and with sufficient robustness to survive through all the processing steps between the content source and the end user. Fingerprinting avoids the robustness and precision placement issues but requires a processing-intensive search support infrastructure.
In general, for ACR purposes, audio fingerprinting or watermarking is preferred over video, insofar as the signals are easier to identify with companion devices using their microphones rather than their Webcams, and, in the case of fingerprinting search matches, there’s less information to process with an audio file. But audio can’t deliver a fingerprint when the track is silent, and there can be synchronization issues with the use of language dubbing in foreign films. Thus, some suppliers are using video fingerprinting as a backup.
Ultimately, the business model may be the determining factor in choosing between watermarking and fingerprinting-based systems for advertising. Watermarking precisely identifies not only the content but the network it appears on, but it also requires the participation of the content provider as well as the advertiser or app supplier. Fingerprinting doesn’t convey where the ad appears but can operate independently of any involvement by content providers.
Still, another startup, TV Interactive Systems, has come up with another approach to ACR which it calls video signature. While it works somewhat like fingerprinting, VS doesn’t look for a unique file segment as required with fingerprinting. Instead, VS identifies “clues” or characteristics of a given piece of content that individually don’t make for an accurate match, but which taken together in batches of about ten per second ensure an accurate match with the database without requiring the more costly processing used for identification in fingerprinting.
Among more recent developments in ACR is the new fingerprinting system offered by Vobile, which for several years has been supplying VideoDNA technology along with a database of authorized video fingerprints, metadata, and business rules from movie studios and programming networks to enable fully automated identification, tracking and management of content. Vobile’s integrated ACR solution will drive novel smart TV applications, such as broadcast monitoring and advanced advertising models, says CEO Yangbin Wang.
“Internet-enabled smart television brings TV entertainment to a new level,” Wang says. “With our latest ACR solution, content owners can track content distribution anywhere, anytime to any device. This opens up a world of new revenue opportunities for television programmers.”
As previously reported (October, p. 10), beyond the mobile device realm one of the key opportunities for ACR on the advertising front is the connected TV. TV Interactive Systems is focused exclusively on this end of the device market.
Startup Zeitera, along with pursuing ACR TV apps for companion devices, is offering its fingerprinting cloud-based service to TV set manufacturers, providing them a means of working directly with advertisers to enable placement of targeted ads in linear programming without requiring any participation by the content suppliers.
Civolution has its eye on the connected TV space as well, Terpstra notes. “We have our technology run on the chipsets within these Web-connected devices to provide the ability to recognize content within the TV sets,” he says.
“Especially with using watermarks, we can determine the precise location of an advertisement that is being played out, and we can recognize which advertisement it is,” he explains. “If you know the exact location and which advertisement it is, you can also replace it with another advertisement that’s a local ad or targeted ad if the user’s profile is also known to the server. We’re working with advertising companies to close the whole loop.”
In its latest move, Zeitera has introduced a software development kit and API development toolkit for programmers, CE manufacturers, and app developers who want to implement the firm’s Vivid audio and video fingerprinting ACR solution in conjunction with companion apps on tablets and smartphones. Zeitera offers its solution as a cloud-based service which is designed to eliminate a lot of the heavy lifting associated with individual users’ implementation of ACR.
“The Vvid Mobile API and SDK Development Platform for iOS and Android fills the needs of those broadcasters and content owners that would like to have smartphone and tablet applications that can synchronize with their TV programs, commercials, and movies,” says Zeitera CEO Dan Eakins. “Zeitera is pioneering the market for white label ACR services and has several partners signed on developing applications with these APIs.”
