At a time when “Nielsen Overnights” have faded amid the declining linear TV business, and this present top gunAside from the holiday weekend, the power of weekend box office reports is also waning, the weekly streaming horse race has quickly become a popular destination to satiate the competitive blood thirst of entertainment consumers.
But the systems in place for determining weekly winners don’t seem ready for prime time.
After months of confusion from various conclusions drawn by Nielsen, Netflix and other companies that publicly report streaming viewership, Next TV tries to delve deep into the making of this sausage.
Also read: Nielsen and Netflix are at odds again over metrics programming
We weren’t allowed into the factories – neither Nielsen nor Netflix wanted to be open with us about how their methods work. But we’ve found enough to report that ultimate “winners” are difficult to determine – it often depends on which methodology is used. And true competitive audience analysis can also be hard to come by.
Like it always was?
In terms of streaming audience measurement, the most prominent voice right now is Nielsen’s Weekly Streaming Top 10 ranking, which began counting viewers for the largest US subscription streaming services in 2020.
Nielsen, an embattled research firm that was acquired by private equity for $16 billion just in March, is trying to maintain hegemony over a television industry it’s ruled for decades but now faces a slew of challengers in the streaming era is.
Each week, Nielsen ranks the top 10 shows for Netflix, Amazon Prime Video, Hulu, Disney Plus and Apple TV Plus based on total watched minutes for across. Nielsen publishes four separate charts: a ranking of the top performers overall, along with sub-rankings for original series, off-net streaming TV shows, and films.
Nielsen pulls this data from its SVOD Content Ratings panel of 41,000 households and accounts for multiple users in the same home watching the same show together.
But there are key limitations. For one, the finite sample is “just glass,” meaning it doesn’t measure viewership across phones, tablets, and computers. Netflix, by far the largest SVOD presence in the US with around 66 million of its 220 million global users based here, won’t say what percentage of its audience watches shows on mobile.
However, a survey conducted by the Leichtman Research Group in 2019 found that around 85% of viewers watch Netflix shows on TVs. But again, conclusions vary – comScore, for example, found back in 2014 that 49% of younger viewers aged 18-34 are watching on some IP device.
Nielsen’s competitive analysis is also skewed by Netflix’s dominance. Unlike broadcast television, where it can be assumed that all viewers in a given market have the same over-the-air access to all local channels, Netflix still has many more viewers than any other SVOD service, measured in Nielsen’s Weekly Streaming Top 10. Therefore, most of the shows that appear in Nielsen’s rankings belong to Netflix.
Meanwhile, its SVOD rankings are missing key ingredients, including HBO Max and Paramount. These services have no agreements with Nielsen to be included in the ranking.
Back in early 2021, this became an issue for the former WarnerMedia, which asked to create a special Nielsen Weekly Streaming Top 10 ranking featuring HBO Max and day-and-date streaming release Wonder Woman 1984topped the Nielsens charts for Christmas week 2020. HBO Max hasn’t been back since.
Nielsen uses audio signature technology, which takes time to count, to measure audience usage in its content rating panel. This is the main reason why the research company takes about 25 days before reporting performance data for any given week.
Numerous research companies are currently competing to provide viewership data across the broader range of SVOD, AVOD and FAST services. media like Next TV receive only a small portion of the iceberg sticking out of the water.
But even in this niche market, the race is competitive. Samba TV, which measures usage on smart TVs, regularly releases household performance data on broadcasts to the press. Reelgood, on the other hand, makes a mobile app that allows users to type in a show or movie to find out all the different platforms it’s available for. And Reelgood regularly posts non-contextualized rankings of shows based on its customer testimonials.
Then there’s Demand Media, which has gained prominence through a proprietary formula that ranks programs based on their “demand.”
But no other third-party measurement scheme has delivered a ranking as clean and as contextualized as Nielsen’s Weekly Streaming Top 10, the Next TV reports weekly through December of last year.
Uh, Houston?
In September, Netflix rolled out the splashy launch of its Global Top 10 ranking, which expanded its publicly wiped out Top 10 in the US ranking widget to include four detailed weekly charts listing its top series and movies, both by English as well as non-English – English language formats, by hours viewed worldwide.
Netflix’s four weekly rating charts provide a clean assessment of the hours streamed on Netflix’s global platform of over 220 million user IDs, including indicators showing the countries where programs have performed best.
Netflix’s Global Top 10 quenches the thirst for immediacy in a way Nielsen doesn’t – the Global Top 10 comes out on Tuesday, 48 hours after the peak of the measurement week the previous Sunday.
There are downsides, starting with the fact that a FAANG company is essentially self-reporting and its rankings provide absolutely no competitive context.
Also, the Global Top 10 counts how many hours of a show were streamed by user IDs – there is no account for co-viewing.
Regardless, after years of being accused of a lack of accountability and transparency, Netflix has finally come up with a hedge.
And despite the limitations of the Global Top 10, it has challenged the accountability and transparency of Nielsen, which had steadily gained traction with its SVOD ranking.
For example, in January, Nielsen declared Disney Plus a day-and-date animated film Encanto the winner of Christmas week’s streaming race, beating Netflix’s Oscar-nominated satire directed by Adam McKay Don’t look up with a significant advantage.
Showbiz traders and enthusiast pubs rushed to explain Encanto the winner over Don’t look up. But comparing the rankings from Nielsen and Netflix made it clear that something was wrong.
Credited to Nielsen Don’t look up with only 521 million US viewing minutes for the week of December 20-26, or approximately 8.7 million hours. Netflix has clocked its show at 111 million hours worldwide. For both Nielsen and Netflix to be right, only 8% of the audience for an English-language film set in America and having American talent came from the US, a market that controls 30% of Netflix’s total audience.
Nielsen didn’t answer Next TV‘s request for comment. But in a footnote in its rating report the following week, the research firm said it was revising that Don’t look up audience by a factor of 3X, which would have made the film the winner of Christmas Week with a bullet.
At this point however Encanto was declared the big winner of a key holiday frame, Omicron-related cinema closures greatly increased the importance of the streaming window, at least in the eyes of the public.
Dealing with the Don’t look up Matter calls into question not only Nielsen’s transparency after a counting error, but also the effectiveness of its audio signature technology.
In April, Nielsen declared the Disney Pixar film To redden the winner over Netflix Ryan Reynolds comedy action film The Adam Projectalthough the comparative mathematics performed by Next TV clearly favored by Netflix.
And just last week, Nielsen explained to Netflix Better call Saul the winner of the April 18-24 subscription streaming race, despite the AMC offnet series not even registering in Netflix’s Global Top 10.
Again, Nielsen executives refused to speak to them Next TV on the plate. However, people familiar with the company’s measurement systems told us that discrepancies Better call Saul could be due to the fact that Nielsen ranks shows by their full episode count, while Netflix ranks them by season.
However using Netflix data even if we cumed all 50 replay episodes of Better call Saul available on the streaming platform, we wouldn’t get close to the April 18-24 audience performance registered by the Netflix original series anatomy of a Scandal, Ranked #1 in Netflix’s Global Top 10, but only #4 by Nielsen.
Sources close to Nielsen add that while the numbers may vary depending on methodology, it depends on the competitive context that the Weekly Streaming Top 10 provides.
But again, with limited participation, transparency and accountability, this claim is suspect.
So for now Next TV will continue to report Netflix’s global top 10 and include other metrics on an ad hoc basis. The public wants – the video business needs – an effective third-party measurement system.
We will continue to monitor this area until one develops.
Varis Van zel 