A small analysis of video quality on HD discs, with a new addendum
As I expected, my column provoked quite a bit of email traffic. The overwhelming majority of comments were very positive, but there were a number of readers who objected, making valid points that should be addressed. For the sake of those who may not have read the original column, we’ll begin there. For those who’ve already read it, please proceed to the addendum below.
When I was particularly impressed with 20th Century Fox’s return to Blu-ray Disc, I noted that the video CODEC for each is AVC. I then vaguely recalled that many of the discs that impressed me were also compressed using the AVC CODEC. So I thought it might be interesting to correlate CODEC type against video quality rating for those discs that I’ve reviewed to see if there was anything to be learned. Since I began reviewing HD on disc in the spring of 2006, I believe I’ve accumulated a sufficiently large enough sample to identify valid trends (assuming they exist).
From this little analysis, I might learn a few other things. I was curious to discover whether or not my ratings were consistent and reliable. Since I routinely watched (and continue to watch) broadcast HDTV, I had a baseline of experience, but HD discs are so impressive compared to DVD that I might have given HD discs in early reviews a higher rating than they deserved simply because I was so initially impressed. If that had happened, after I became accustomed to the look of HD discs, my ratings would have dropped off.
I was also interested to see if, as the production houses became more accustomed to the compression tools, and as advanced CODEC tools came on line, if there had been an improvement in the quality of the transfers.
But perhaps the most contentious question is whether there is any statistically noticeable numerical difference in the video ratings between HD DVD and Blu-ray Disc. As you can imagine, I’ve received quite a few emails from readers who disagree with my expressed preference for Blu-ray Disc, which I find offers a more film-like presentation. I’ve written that the best of BD looks better than the best of HD DVD and I’ve suggested that the differences in bit rate and storage capacity are the technical reasons why. As a result, I’ve been accused of blatant prejudice.
So I gathered the video ratings and CODECs from all my reviews, dumped the data into an Excel spreadsheet, and began to analyze. And the results are . . . interesting.
A plot of the raw data is difficult to read because of the variability, so I calculated a moving average with a sample size of three, but although the curves became a lot easier to read, the plotted values are distorted. I also ran a linear regression of the data for each format. To put technobabble aside, a linear regression is simply a way of fitting the numerical data to a straight line approximation. The slope or tilt of that line represents a trend.
The resulting graph reveals several interesting things. First, for the most part, BD seems to have scored higher than HD DVD in video quality. Second, the trend analysis seems to indicate that BD is improving (the trend line indicates that during the review period its score improved by 0.76 video points), while HD DVD had a slightly negative quality trend (losing 0.12 video points on its trend line). But Blu-ray Disc’s quality trend may not be an indicator of a dramatic improvement in the skills of compression houses; it’s more likely a consequence of the early use of MPEG-2 and single layer discs. Once Blu-ray Disc’s capacity doubled and advanced CODECs dominated, such a dramatic improvement should be inevitable. It’s nice to see that the analysis demonstrates an expected result.
Now, one could argue that I gave HD DVD lower ratings from bias, but I can assure you that’s not the case. I simply report what I see. One could also argue that my HD DVD scores dropped slightly as I experienced the video quality of Blu-ray Disc. That is possible. Regardless, the differences in my perceptions of the two formats remain valid. So even if HD DVD had improved over time, Blu-ray Disc must have improved more.
MPEG-2 was used for early BD releases and was used for a small handful of HD DVDs, but the real CODEC workhorses for HD on disc are VC-1 and AVC. And when I calculated the average video ratings for BD and HD DVD for both CODECs, here’s what I found:
Several findings are of particular interest. First, regardless of the format, the AVC CODEC seems to produce a more pleasing presentation than the VC-1 CODEC. Second, regardless of CODEC, Blu-ray Disc seems to offer a more pleasing presentation than HD DVD. Third, the differences between the appearance of transfers compressed with the two CODECs are more noticeable on Blu-ray Disc than on HD DVD, suggesting once again that BD is more revealing. Admittedly, the original data points are based on purely subjective assessments, but I’m confident that the data are representative of accurate observations rather than bias.
When DVD was young and some studios were issuing discs in non-anamorphic video, there were respected reviewers who took the position that since most people did not have displays capable of the anamorphic squeeze that non-anamorphic discs should be the norm. Those writers objected to anamorphic DVDs because they were reviewing discs on non-anamorphic displays; for a 4:3 display, DVD players throw away every third scan line of an anamorphic transfer to restore proper proportions, which reduces video resolution. What they saw was real, but theirs was an incredibly short-sighted position. When those same reviewers purchased displays capable of the anamorphic squeeze, they suddenly flipped their positions. They became anamorphic video advocates and complained whenever a studio resurrected a non-anamorphic laserdisc master for a DVD release. What, you may ask, does this have to do with the subject matter at hand?
My point is this. I recommend that consumers support the highest possible quality HD disc format to ensure that when they eventually purchase a sufficiently revealing display, the quality of the discs will maximize the enjoyment factor. Consumer electronics (and PCs) have a wonderful tendency to offer more for less over time. Average HD-ready display prices are dropping. Average HD-ready display performance is improving. Within five years, most of you will purchase displays that will reveal the subtle differences between Blu-ray Disc and HD DVD. I humbly suggest that your dollars should be invested in the format that is more able to deliver a viewing experience that comes closest to the motion picture theater experience.
And I would be remiss if I didn’t suggest to the studios that if the AVC video CODEC produces better results, perhaps that would be the better choice for future releases.
Some HD DVD proponents will interpret this article as prejudiced support for Blu-ray Disc. In response, I can only write that I report simply what I observe, and I have no vested interest in either format other than a desire to maximize my and your enjoyment of home theater. Let the flames begin.
Some readers still don’t accept my suggestion that BD produces a better presentation than HD DVD. One wrote to accuse me of bias, would not believe the technical reasons that BD is potentially a better format than HD DVD, said that he could no longer trust me, and he would leave the site forevermore. I wished him well and further hoped that he could find a site that didn’t share my “bias.” For that writer and others who expressed a desire for a more inclusive statistical analysis, one that would include other sites, I’ll offer external data. I’m reluctant to report these scores since the displays used for the reviews range from direct view CRTs to LCD flat panels to a very few high-end displays. So I’m not confident in the quality of the displays, nor, with one exception, am I familiar with the judgments of the writers. But, I think it’s safe to assume that differences between the two formats may be valid, even if I’m not comfortable with the absolute scores. So by popular demand, here are some numbers based on many hundreds of reviews from other sites. (Notes: the scores were normalized to our zero to ten scale; and, the greater BD to HD DVD differences were earned on better displays.)
Average Video Score BD
Average Video Score HD DVD
Home Theater Spot
Home Theater Forum
Average of the Averages
Please note that every single site scored BD video quality higher than HD DVD. So if I’m biased, I guess I’m not alone.
Some readers observed correctly that while the BD camp implemented advanced video CODECs and abandoned the legacy CODEC, in the HD DVD camp Universal seemed to be cranking out quantity rather than quality. So, readers pointed out that the shape of my BD curve was due to the abandonment of MPEG-2 and the implementation of AVC. All true, but I suggest that this does not alter the overall video quality ratings. Let’s first limit our consideration to two very recent blockbuster releases that were both compressed with the AVC video CODEC and should have received very careful transfers: Paramount’s Transformers and Fox’s Fantastic 4: Rise of the Silver Surfer. (Once again, the scores have been scaled to our zero to ten video rating.)
HD DVD Video Score
BD Video Score
Please note that for these very recent and very carefully prepared HD discs, BD outscored the HD DVD.
But we can expand our view beyond two titles. Limiting myself to DVDfile’s video scores, let’s take a look at the ten discs in each format that were compressed with the AVC video CODEC and earned the highest video ratings in our review database:
Percentage of discs with video score of:
Please note that BD outscores HD DVD; in fact, all the best BD scores are greater than the best HD DVD scores.
Now, let’s take a look at the top eight discs in each format that were compressed with the VC-1 video CODEC (I couldn’t manage ten each since I only have eight BDs compressed with VC-1).
Percentage of discs with video score of:
Please note that all these BDs compressed with the VC-1 CODEC are from Warner Home Video. So these transfers were compressed for the space limitations of HD DVD and then re-authored for BD. This is a difficult comparison to interpret. However, please note that the highest VC-1 score in either format is lower than the highest AVC score for BD. This reinforces my previous suggestion that the AVC CODEC produces a more film-like presentation that the VC-1 CODEC.
Several readers have written to ask which titles in each format I consider the best I’ve reviewed. They expressed the wish to make the comparison themselves to see if they can reproduce my findings or refute them. I’m about to list those titles, but before I do, I must remind everyone that not all players and displays are equivalent. Both print magazines and websites devoted to hardware reviews routinely evaluate and score visual quality in both sources and displays. Their experiences and my own are similar. Some components are not as revealing as others. So with that in mind, here are my top picks for video quality:
HD DVDs in alphabetical order (the discs that scored 9.5):Aeon Flux
Chronicles of Riddick
One Six Right
Phantom Of The Opera
One was compressed with MPEG-2; the rest were compressed with VC-1.
BDs in alphabetical order (the discs that scored 10):Black Book
Hitchhikers Guide to the Galaxy
Pirates of the Caribbean
Fantastic 4: Rise of the Silver Surfer
One was compressed with MPEG-2; the rest were compressed with AVC.Departing Thoughts
Not final thoughts, simply departing thoughts . . . I’m sure the debate is not over.
The one aspect I could not yet find time for is a multi-site analysis of how the CODEC affects the video scores. That may have to wait for a later time.
And I’m still baffled by the few readers who cannot accept the technical explanation that a higher bit rate and less compression for any given CODEC will produce a better image. Sony’s Superbit DVDs demonstrated very clearly that less compression and a higher average bit rate produces a better looking video transfer. Why doesn’t that experience with MPEG-2 convince skeptics that similar improvements will occur with VC-1 and AVC on HD discs?
But for now, I hope this enhanced analysis answers most of your questions.