A question of quality…

Update: I’ve re-tested Resolume with the latest available update – 3.0.1 at the time of this writing, and some results have changed:

So there was a question on the VDMX forum regarding output quality of various VJ softwares, and questioning which is best. I decided to do a little test of Resolume Avenue, Modul8, VDMX and v002 (why not..?) compared to Quicktime player for quality, just out of curiosity. What I found was rather interesting for a variety of reasons. I wont pretend to be an expert at any of these softwares, so bear with me. If I make obvious mistakes, point them out!

Testing:

OMGWTF LOGOs

My test clip is a rendered out from Final Cut Pro 6.0.5 clip of bars (and tone..), with time code and my logo (because im like that). It may not be the best real world clip to use, but the choice mistakenly lead to an interesting revelation. Almost all commercial VJ software fucks up on handling non 1:1 pixel aspect ratios correctly, and none get the color right. DVCProHD is actually, if you are counting, 960×720 pixels, but the image is displayed at 1280. This apparently is enough to through off most Vj apps.

Quicktime Player

This is our reference. Lets jot some quantitative color values down. (I grabbed these from the Digital Color Meter in Application/Utilities)

  • Light Grey: 186,186,186
  • Yellow: 181, 193, 0
  • Cyan: 0,192, 198
  • Green: 0 198, 0
  • Magenta: 219, 0, 198
  • Red: 215, 0, 0
  • Blue: 34, 0, 199
  • Purple: 71, 0, 120
  • White: 255, 255, 255
  • Black: 0, 0, 0
  • Pluge (dont ask), 7, 8, 8

Now, those values (if you are a video nerd…) seem whacky. Bear with me (or jump to the conclusion…). Otherwise aspect ratio is correct, and everything looks and plays fine as expected.

VDMX

VDMX

The first thing I did was make sure my output settings were correct for the DVCProHD, and set them to 1280 x 720. Made a new project, a new media bin, a new layer, and loaded in my clip. The aspect ratio was definitely off, and I could not really get VDMX to display it at a proper 16×9 without going to ‘stretch’ mode, which seems, to me, like a bug (reported).

Color wise, we are not to far off, but not on the ball:

  • Light Grey: 179, 179, 179
  • Yellow:179, 179, 0
  • Cyan: 0, 179, 179
  • Green: 0, 179, 0
  • Magenta: 179, 0, 180
  • Red: 179,0, 0
  • Blue: 0, 0, 179
  • Purple: 47, 0, 101
  • White: 255, 255, 255
  • Black: 0, 0, 0
  • Pluge: 5, 5, 5

Hrm. Interesting. Thats not quite right (according to our reference). Video quality seemed fine, even though the aspect was off a bit. I did not notice any weird edges or odd artifacts in the image, but its hard to say without it being displayed with the right aspect ratio. David Lublin suggested I try various blend modes because VDMX can leverage raw OpenGL or Core Image pipelines internally. Switching between OpenGL Over and / Source Atop blend modes made no difference (which is good if you think about it). Setting the layer position mode to stretch looked just like QT.

Modul8

Modul8

Now, getting Modul8 to actual display my clip properly was a real pain in the ass, I had to manually adjust the scale of the clip in the transform controls, as I could not get it to display correctly at scale 1.0 (or.. maybe thats 2.0 x in Modul8 lingo? I have no idea, the UI says 2 which is what it started with launching the app. How that makes sense I have no idea) . To actually fully display (it was being clipped on output initially, see the screenshot), I had to adjust the transform scale to somewhere between 1 and 2. However, once I adjusted the scale to see the full extent of the image, the aspect ratio was indeed correct. And the color, dead on with VDMX’s ‘interpretation’ of our quicktime movie.

  • Light Grey: 179, 179, 179
  • Yellow:179, 179, 0
  • Cyan: 0, 179, 179
  • Green: 0, 179, 0
  • Magenta: 179, 0, 180
  • Red: 179,0, 0
  • Blue: 0, 0, 179
  • Purple: 47, 0, 101
  • White: 255, 255, 255
  • Black: 0, 0, 0
  • Pluge: 5, 5, 5

Qualitatively, the image looks softer, with some blurring/issues on the edges of diagonal lines, especially the text/timecode readout and my v002 logo. It looked almost as if the image had been re-sampled. You have to look pretty hard to see it, but it makes me think video was being scaled or processed at a set resolution. Thats speculation, I have no idea about the internals of Modul8. Its not huge, but it looks like its there.

Resolume Avenue

Avenue

Edit: Pixel aspect ratio handling is fixed in Resolume 3.0.1., and the color results have changed: see the addendum at the end of the Resolume Avenue section

It took me some time to figure out how to set the current ‘Composition’ to 1280×720, and when I did, I was presented with something far stranger than I had anticipated: It not only messed up the pixel aspect ratio, but decided to fill in the rest of the missing space with the right hand side of the bars blue:

Avenue

Color wise…

  • Light Grey: 161, 161, 161
  • Yellow: 161, 161, 0
  • Cyan: 0, 161, 161
  • Green: 0, 161, 0
  • Magenta: 161, 0, 162
  • Red: 161, 0
  • Blue: 0, 0, 161
  • Purple: 29, 0, 83 (looks more like dark blue here..)
  • White: 255, 255, 255
  • Black: 0, 0, 0
  • Pluge: 0, 0, 0

The whole image was much darker, and seemed shifted. The lack of pluge means you loose definition / detail in your blacks. I dont know if this is indicative of using DVCProHD, but its not necessarily a good sign… the shift in the blue also means something else weird is going on. Perhaps they are using a custom YUV ->ARGB conversion path with different constants than DVCProHD expects (ie, HD Rec 709 if I recall).

Update to Resolume 3.0.1

Resolume 3.0.1 fixes the pixel aspect ratio issue, so thats moot. Good news is also either I was incorrect in my testing or 3.0.1 also changes some color output slightly. Either way the new values for the same DVCProHD file are :

  • Light Grey: 179, 179, 179
  • Yellow: 179, 178, 0
  • Cyan: 0, 179 ,178
  • Green: 0, 179, 0
  • Magenta: 179, 0, 180
  • Red: 179, 0
  • Blue: 0, 0, 179
  • Purple: 48, 0, 100
  • White: 255, 255, 255
  • Black: 0, 0, 0
  • Pluge: 5, 5, 5

This is much better than previous versions, and the pluge results indicate that we dont miss out on details in our blacks, which some may find important. Good news, and puts Resolume Avenue at almost exactly the same values as VDMX and Modul8. Thats kind of amusing, but I guess this tells us the Quicktime API makes sure of this and no one is doing anything too crazy behind the scenes pipeline wise that would effect the color output.

v002

Ok, so, im jumping the gun a bit but this is going to be my open source VJ app, and the UI is not finished so no pictures. Anyway, it handles the aspect ratio correctly because I happened to load a DVCProHD movie a few weeks ago and noticed it doing what VDMX did, seemingly playing it back at 960×720 rather than 1280. In Quicktime 7, this is called your frame aperture mode, if you set it to presentation or production your issues tend go away, presentation being the preferred for a variety of reasons. Read here for more info.

Color wise, it shows exactly the same colors as Quicktime Pro does:

  • Light Grey: 186,186,186,
  • Yellow: 181, 193, 0
  • Cyan: 0,192, 198
  • Green: 0 198, 0
  • Magenta: 219, 0, 198
  • Red: 215, 0, 0
  • Blue: 34, 0, 199
  • Purple: 71, 0, 120
  • White: 255, 255, 255
  • Black: 0, 0, 0
  • Pluge, 7, 8, 8

And the image seems identical to Quicktimes as well. Yay for me and shit.

Trust No One (except…)

Wait. How do we know Quicktime Player is the “right” reference? Those colors its displaying are the definition of weird. Those Bars are motherfucking COLOR BARS. THEY ARE REFERENCE. THEY ARE LIKE THE WORD OF GOD. NO ENGINEER WORTH HIS SALT WOULD PICK 181, 193, 0 FOR YELLOW. What the shit is going on?

Well, of course, when in doubt, blame Apple. Quicktime uses Colorsync to display and output its video, and not the proper video gamma of 1.8 with proper video color conversion matrices. If you’ve installed Final Cut Studio and open Quicktime Player preferences, it adds a checkbox to allow for Final Cut Pro Color Compatibility, which ‘skips’ over Colorsync and uses Final Cut Pros color correction. Apple giveth. The faith is restored, for $1500.

QT Color with proper 1.8 Gamma and FCP compatibility enabled:

  • Light Grey: 191, 191, 191
  • Yellow: 191, 191, 0
  • Cyan: 0, 191, 191
  • Green: 0, 191, 0
  • Magenta: 191, 0, 192
  • Red: 191, 0
  • Blue: 0, 0, 191
  • Purple: 65, 0, 120
  • White: 255, 255, 255
  • Black: 0, 0, 0
  • Pluge: 11, 11, 11

Goddamn it. It looks like no one correctly handles color properly. I just wasted your time didn’t I?

Final Thoughts

Something I noticed when testing these apps is, at one point, I had all of them open and playing video (VDMX, Modul8, Avenue, v002 and Quicktime), and VDMX consistently had higher frame-rate playback than any other app, including Quicktime Player. I could tell because I had incrementing time code. Thats huge.

Regarding the color accuracy and image quality, this is not something everyone is going to see, or notice, because firstly NO ONE USES DVCPROHD as a playback codec. I just happened to choose it since I had some bars on my drive, and then noticed the aspect ratio issue in some apps, so it piqued my curiosity. PJPEG, MJPEG, AIC, DXV etc all use a pixel aspect ratio of 1.0 so this is why no one has noticed. The only issue here is the component YUV to ARGB conversion, and the differences noted in VDMX and Modul8 are indicative of a lossy YUV->ARGB. Ask any seasoned After Effects user, they have seen this for some time.

Secondly, almost everyone is playing out to non-color calibrated monitors, or even worse, projectors, which are fucking impossible to get any real calibrated color out of unless you have a huge budget and lots of time on your hands. I think the lesson here is this is basically of a non-issue, and I only stumbled on some bugs in some apps handling of odd pixel aspect ratios. Try not to care about all this, and make with the pretty… I feel like I just wasted a lot of time.

13 Responses to “A question of quality…”

  1. moongold Says:

    As an old, now sadly extinct, friend of mine never tired of quoting “Time spent in reconnaissance is never wasted.”

    I read your piece all the way through, kept me entertained for ten minutes, and who knows I may use the knowledge gained in a future.
    One never knows what is going to be useful, sometimes just the act of doing can create a use.

    Big thanks Vade for all your visual related activity, I use your plug-ins all the time, watch your explorations and creativity with interest and now I can applaud your wasted time…

    It wasn’t wasted, but hey, I agree ‘Make with the pretty’! :-)

  2. dan Says:

    Up with pretty! I’ll read any technical discursion no matter how dense if it’s written in your inimitable style, vade. What the shit indeed.

  3. Christopher Wright Says:

    Holy crap — thanks for writing this up. I _hate_ how poorly color handing is in OS X (and how _every_ core technology does it differently; CoreImage != QuickTime != Quartz2D != QuartzComposer != OpenGL != CoreGraphics…. it’s a nightmare). And of course anyone who actually notices color shifts _Craps Their Pants Incessantly_ about it _All The Time_ …. I hate writing graphics software. Just give me my freaking colors, unadulterated! :)

  4. toby*spark Says:

    thats interesting about vdmx vs quicktime player: in my forays into the high-end events world, that the vjs apps couldn’t match the perfectly smooth playback of quicktime, and it *was* noticed. i put this down to their on-demand rendering nature. mmm…….

  5. vade Says:

    Chris – Exactly. Its a huge issue, and it seems like there are no unifying docs to help show how things tie together between technologies. Its a real issue, and is actually a huge performance issue as well because a lot of Apple libraries do extra conversion behind the scenes for you, when you are trying to handle it yourself.

    It sucks. And indeed, I also hate writing graphics software!

    Toby – I was not testing the individual apps for smoothness, but just noting that under high load, VDMX seemed to be able to not drown as fast as the others. I definitely have seen QT player, on its own, playing a huge file result in smoother playback than other apps playing the same huge file. Is that what you mean?

  6. erik Says:

    hej. i also enjoyed the article.

    a little note for resolume users: i use the picvideo mjpeg codec and i have a lot of videos recorded with dvcprohd. when i tested it the video was simply in the middle of the screen with black on both sides. in the clip properties you can set the width manually to 1280.

  7. Bart Says:

    Great test vade! Could you send us your test video so we can debug it? We,d love to take a look.

  8. Bart Says:

    Thank you for sending us your test footage. You used an old Avenue beta (rev990) for your test. Version 3.0.1 handles various pixel aspect ratios (including this movie) just fine.

  9. Bart Says:

    About the colours, In the screenshot I see you’re playing the video with the layer opacity set to 50%. Of course you are going to get completely wrong colours if you do that. Please set the layer to 100%.

    Also, when measuring with the DigitalColor Meter I even get different results depending on what screen I measure connected to the same computer.

  10. vade Says:

    Thanks Bart – I just tested 3.0.1 and indeed the pixel aspect ratio stuff is fixed. Thanks, Ive updated the article to mention that.

    As for the color, The image I used was just a reference image to have a photo, I just re-ran color tests with 3.0.1, and with the DVCProHD movie the color output is almost exactly the same as VDMX, so either something color wise changed in 3.0.1 or I did indeed mess up in testing resolume initially. I do indeed recall setting the opacity correctly, but either way I’ve updated the article and have added an addendum section specific to 3.0.1.

    As for Digital Color meter, I understand it to measure color pre display correction to give you true values as expected digitally (pre display munging), so you should see the same values across displays. I do, in testing my Samsung 17″ display attached to my MBP gives me the exact same results for Resolume.

    In fact, I just confirmed it. I ran the Colorsync profiler application and changed the values while watching the digital color meter tell me it was reporting the same exact values regardless of gamma curve or profile modification. This makes sense; Taking a screenshot with multiple displays does not distort an image due to Colorsync calibration differences for the output devices. A red desktop across 1 display will look the same on another in the screenshot (as viewed on a single display), perhaps not to your eye, as the display device may display the same digital red values differently due to different input responses and output responses between the devices – which is the point colorsync tries to solve – match the internal digital representation with the output devices actual physical representation as close as possible.

    Regardless : testing on the SAME display does make sense because your baseline calibration profile does not change on a test by test basis, as all tests are done within the same base “offset” or environment, which is how I tested originally :)

  11. bart Says:

    Thanks for the update vade! I really appreciate it.

  12. Ilan Says:

    Some how this slipped by me. Well… I am not a very highly technical person. Quite a few errors in the writing but otherwise a fun and intriguiging journey of a highly methodical investigation. I think this kind of research/feedback is highly valuable and I will pass this info on to others in the field who like me may not have been paying attention.

    We’ll be watching.

    And thank you.

  13. vade Says:

    Hey Ilan, let me know what the errors are, I want to update the post if If there is anything glaringly wrong, or a new version fixes any issues. Let me know!

Leave a Reply