Stretched Video and Nonsquare Pixels

As you might remember from a previous post I've been playing around with the new iPlayer service from the BBC. This is a great service allowing you to catch up on TV you missed in the last 7 days. The available content is limited to stuff the BBC has the distribution rights for, but even so I can still watch a lot of TV this way. Also it gives access to regional programming so I can watch things that are only broadcast on BBC Scotland or BBC Wales which I couldn't have watched before. Basically I'm a fan! This post, however, isn't about the iPlayer but rather about some weird behaviour I came across while using it and a solution.

The programmes available through the iPlayer are provided as Windows Media Video (WMV) files. The current version is encoding the programmes using a technique known as nonsquare pixels. Computer images are usually encoded with square pixels, i.e. the sampling rate is the same horizontally as it is vertically. With nonsquare pixels there tends to higher sampling in the vertical dimension. This means that if you look closely, the picture looks like a brick wall rather than a square grid of pixels. In WMV files this is achieved (I think) by stretching the image to be encoded so that it fits the square grid and then when it is played back squashing the square pixels back to their intended shape. The files from the iPlayer have a pixel aspect ratio of 16/11 which means that each pixel is about half as high as it is wide.

Assuming the pixel aspect ratio is correctly encoded in the video files (for those interested in the technical details there are two metadata entries AspectRatioX and AspectRatioY that specify the values used to squash the square pixels) then the file should play back as intended. If the pixel aspect ratio values are missing (which I have seen in a few cases) then the effect is similar to that seen when widescreen (16:9) content is stretched to fill a traditional (4:3) TV screen -- everyone looks tall and thin. If you are seeing this problem and you think it might be missing metadata then you can use the WMVARChanger tool to add the metadata entries to the file. Missing metadata is a rare occurrence yet it seems that a number of people still see stretched content.

I'd heard a number of people complaining that files they were trying to watch were stretched and unwatchable, yet I knew that there was no problem with the files themselves as I'd also downloaded the same programmes and watched them without issue. At this point I started to try watching files encoded with nonsquare pixels on as many different computers using as many different versions of Windows Media Player as I could find. Fortunately this is quite straightforward for anyone to do as Microsoft kindly provides a number of different nonsquare pixel test files. In all but one instance the file played back correctly and looked as it does in the first image below. But in one case I got stretched content as you can see in the second image.




Given that the white shapes are meant to be perfect circles you can see that the first image is fine, whereas the second image is stretched vertically distorting the image. Now I don't know exactly why the problem occurs but I do know the culprit and how to fix it.

It turns out that when the problem occurs disabling video acceleration (via the Performance tab of the Windows Media Player Options dialog box) fixes the problem and the file is played back at the right size. In fact you don't have to fully disable video acceleration just disabling the "Use video mixing renderer" in the advanced options seems to do the trick. I'm assuming that the problem is actually the video driver, that is either ignoring the pixel aspect ratio settings or is simply unable to display nonsquare pixels. Either way at least there is a (albeit nasty) solution.

If you really want to reproduce the problem yourself then you can. It turns out that the video acceleration provided by Microsoft Virtual PC suffers from the problem. So if you setup a Windows XP virtual machine and then try the sample files you will see that it does not behave correctly (tested with a standard Windows XP and SP2 installation).