Open Frameworks + Quartz Composer

Yesterday Mary Ann and I went to Parsons to take part in an Open Frameworks knitting circle. I’ve been meaning to get into OF for some time, especially since Memo has been working on his Cocoa tools for OF (native Cocoa windows and NSOpenGL, etc). After seeing his work and others work in the OF scene, it seemed only natural to want to leverage the libraries and add ons that OF has within QC, because frankly the work is awesome.

Zach Lieberman was kind enough to walk me through the basics of the OF architecture and how the classes and drawing code talks to one another. After some library linking stumbling blocks Open Frameworks now runs within a QC plugin.

How it works is this:

You program your OF ofBaseApp (usually testApp.h & testApp.cpp) as you normally would, and implement all the functions such as setup(), update(), draw(), mouseXXX etc like any other OF app. Rather than implement a main.m like usual and instantiate a GLUT window (or whatever), we will have Quartz Composers plugin instance handle loading our test app. and we have a proxy window class that handles virtual input from our Quartz Composer plugin. The ofxQCBaseWindowProxy is a simple ofAppBaseWindow class I made that allows your test app to be see a “window” even tough you don’t have access to one in Quartz Composer land (you are only handed off an OpenGL context by the parent app loading QC, no window pointer or anything of the sort). We handle mouse (and eventually keyboard) input to our “window” from Quartz Composer.

Once our new OF powered plugin inits we simply create a new ofxQCBaseWindowProxy, and call our normal ofBaseApp initialization stuff:

		windowProxy = new ofxQCBaseWindowProxy();
		ofSetupOpenGL(windowProxy, 1024, 768, OF_WINDOW); 
		ofRunApp(new testApp());

		// we have to manually run setup()
		ofGetAppPtr()->setup();

The dimensions of the window set up the assumed coordinate space of the Open Frameworks calls you make in draw(). Quartz Composer uses a different coordinate space (-1 to 1 for width and -aspect ratio to aspect ratio for height). This lets us draw like we would using GLUT and not have to re-code all of our assumptions. It also lets us use getWidth() etc and have it return sane results.

Now we have to handle a draw loop and event loop, but really Quartz Composer has methods called every frame, so we just use that and manually call setup (via our window proxy) once right after init, and call update and draw using our window proxy within Quartz Composers executeAtTime: method. We expose mouse input via regular Quartz Composer plugin input ports, and can pass in any data we want (real mouse or virtual) for mouse coordinates and button states. Keyboard handling is not up and running yet, but is possible in a similar way.

The basic executeAtTime method looks something like:

	CGLContextObj cgl_ctx = [context CGLContextObj];
	CGLSetCurrentContext(cgl_ctx);
	CGLLockContext(cgl_ctx);
	
	// handle virtual input to our of testApp Pointer
	if([self didValueForInputKeyChange:@"inputMousePositionX"] || [self didValueForInputKeyChange:@"inputMousePositionY"])
	{
		float normalizedMouseX = (self.inputMousePositionX + 1 * 0.5);
		float normalizedMouseY = (self.inputMousePositionY + 1 * 0.5);
		
		float mouseX = normalizedMouseX * windowProxy->getWindowSize().x;
		float mouseY = (windowProxy->getWindowSize().y) - (normalizedMouseY * windowProxy->getWindowSize().y);
		
		ofGetAppPtr()->mouseMoved(mouseX, mouseY);
		ofGetAppPtr()->mouseX = mouseX;
		ofGetAppPtr()->mouseY = mouseY;
	}
		
	// do our actual per frame drawing here.
	windowProxy->update();

	windowProxy->draw();
		
	CGLUnlockContext(cgl_ctx);

Really thats it for the basics. The only gotchas are OpenGL in OF not liking CGLmacros so you loose some speed by manually calling CGLSetContext. The only other things that need tweaking are the data path for loading resources, ( easily done via ofSetDataPathRoot() ), testing addons, and making sure each OF QCPlugin we make has a unique plugin identifier (because you know we will have more than one… ).

Now, things get a bit more complicated if you want to pass in images to Open Frameworks from QC. This is totally doable, but can’t be done easily, so ill eventually supply an ofxQCImage class that will allow a for an ofImage to be initted from a pre-existing texture. Passing images out of QC should be no problem as there are built in provisions for making QCImages out of existing textures (which you can get out of an ofImage).

I’ll try and provide 3 example projects shortly that demonstrate how to use Open Frameworks to create a simple Consumer plugin (rendering directly to the OpenGL world), a simple Provider plugin (typically outputting an image for Quartz Composer to be able to use elsewhere), and a simple Processor plugin which takes in an (input image and outputs an image – think video effect).

Just to get an idea of how awesome this is, Here is the OF QC Plugin running within VDMX. No extra work was needed to be done. You get all of this inter-application capability simply by being usable within Quartz Composer. This means After Effects compatibility with 3rd party wrappers, iChat, PhotoBooth, Final Cut, the list goes on.

VDMX + OF (via QC… lots of acronyms!)

And being post processed by VDMX with my blurs:

10 Responses to “Open Frameworks + Quartz Composer”

  1. toby*spark Says:

    so happy to see this happen. doing this was exactly my thoughts after going to an OF workshop… just i didn’t have the balls to actually try.

  2. Louis Muloka Says:

    très cool.

  3. scott Says:

    i like this very much. my hat off to you, anton.

  4. joshua goldberg Says:

    you are a god.

  5. thomas traum Says:

    THANKS!!

  6. Bruno Gustavo Costa Says:

    this is amazing. i always thought someone would do the opposite first.

  7. alx Says:

    I was wondering when someone was going to get around to this. I was hoping it might be me, but as ever, other things got in the way…

    Great work Anton!!

  8. underdoeg Says:

    This looks really really interesting. I’ll definitely check it out. Great work

  9. vade Says:

    Yeah yetkin there are some examples on the google code page in the download. Thanks for the nice comments. Its still a work in progress but i think it could be useful for some things.

  10. skynoise » Nov Eyeball Snippets Says:

    […] Via Hungary : As well as the free open source VJ software CoGe, the http://coge.lovqc.hu/forum also offers two useful quartz composer plug-ins for real-time compositing. CoGePSBrushes is a free and open-source Quartz Composer plugin, which enables photoshop brushes to be used within a quartz composition. And CoGePSDLayers is another Quartz Composer plugin, which allows separated photoshop layers to be played with inside Quartz. Real-time animation. Via U.S. : You like to code with Open Frameworks? Thanks to Vade, your OF code can now swim happily within Quartz composer. […]

Leave a Reply