Saturn Never Sleeps – studies

January 14th, 2010

Some samples of new footage shot on 7D & GH1 for SNS, mixed in VDMX with smatterings of glitch and optical flow, plus the VDMX workspace incase anyone is interested how I use VDMX (or that im too lazy to crop images):

Saturn Never Sleeps

January 11th, 2010

Upcoming shows this week in Philly and Brooklyn NY. Looks very promising.


Electric Zoo video

January 11th, 2010

I finally got around to editing some of the footage from the Electric Zoo festival Josh Goldberg and I played labor day 09. View it in HD on Vimeo.


v002 Glitch Jpeg

December 25th, 2009

v002 Glitch Jpeg from vade on Vimeo.

Realtime Jpeg corruption on input video frame or 3D rendering in Quartz Composer.

Pass in an image and it is re-compressed on the fly to a corrupted jpeg, and rendered out. Still major work in progress.

Glitch Image Reader

December 24th, 2009

v002 Glitch Image Reader from vade on Vimeo.

Thu Dec 24 01:47:26 Berlin-2.local Quartz Composer[8039] : Corrupt JPEG data: bad Huffman code
Thu Dec 24 01:47:26 Berlin-2.local Quartz Composer[8039] : Corrupt JPEG data: bad Huffman code
Thu Dec 24 01:47:26 Berlin-2.local Quartz Composer[8039] : Corrupt JPEG data: bad Huffman code
Thu Dec 24 01:47:27 Berlin-2.local Quartz Composer[8039] : Corrupt JPEG data: premature end of data segment
Thu Dec 24 01:47:27 Berlin-2.local Quartz Composer[8039] : Corrupt JPEG data: bad Huffman code
Thu Dec 24 01:48:02 Berlin-2.local Quartz Composer[8039] : Corrupt JPEG data: premature end of data segment

Real image de-compression glitch, dynamically generated from stills, in realtime.

Glitch Architecture

December 22nd, 2009

Building constructs in space from random chucks of offscreen graphics memory.

Works in progress: Bokeh

December 4th, 2009

I’ve been fiddling with some new plugins/effects that are in constant state of change. Not everything (ok, almost nothing) I work on gets polished enough for a release, or ever gets to a point where its remotely usable even for me. Add the changes that 10.6 brings with QTKit, 64 bit API changes etc, and this means a lot of things that were ready to go pre-10.6, are now in a weird state of upheaval and need serious attention. Im looking at you DV output plugin and v002 Movie player 2.0…

However I do try and experiment with some new things and get back to some basic fun image processing & effects.

Bokeh

I’ve always been inspired by cinema and photography. One of the things I initially wanted to do with the v002 Blurs was make a nice lens style blur. This is very hard, as lenses have all sorts of interesting caustics and shaped artifacts. That does not lend itself to common blurring methods like convolution at least not for deep blurs with shaped kernels. If you want to add bokeh, it gets even harder as they only appear where there is enough light, and they are much larger than a few pixels across. One method I inadvertently stumbled on in experimentation is called ‘point splatter’ or ‘point splatting’. The basic idea is you render your image not as pixels, but as point geometry with textures. Your texture is your shaped ‘circle of confusion’ or iris/bokeh effect. This ends up code wise doing similar things that the Rutt Etra does, except sans ‘Z’ extrusion, and with luma based ‘size’ modulation. OpenGL nerd talk aside, the effect is fairly convincing:

Compare an original image focused, unfocused camera blur and bokeh to v002 bokeh and gaussian blur. Original camera reference images from CGEducation

30 orbits.

October 24th, 2009

When can I get off this rock?

ofxQCPlugin alpha now available

October 21st, 2009

Just a quick note, i’ve put the Quartz Composer & Open Frameworks bridge code up on Google Code

And there is a thread concerning it on the Open Frameworks forum here

I’ve also included 3 basic plugins, and am adding ports of Open Frameworks examples just to test the water and make sure everything works as expected.

Have fun!

Open Frameworks + Quartz Composer

October 11th, 2009

Yesterday Mary Ann and I went to Parsons to take part in an Open Frameworks knitting circle. I’ve been meaning to get into OF for some time, especially since Memo has been working on his Cocoa tools for OF (native Cocoa windows and NSOpenGL, etc). After seeing his work and others work in the OF scene, it seemed only natural to want to leverage the libraries and add ons that OF has within QC, because frankly the work is awesome.

Zach Lieberman was kind enough to walk me through the basics of the OF architecture and how the classes and drawing code talks to one another. After some library linking stumbling blocks Open Frameworks now runs within a QC plugin.

How it works is this:

You program your OF ofBaseApp (usually testApp.h & testApp.cpp) as you normally would, and implement all the functions such as setup(), update(), draw(), mouseXXX etc like any other OF app. Rather than implement a main.m like usual and instantiate a GLUT window (or whatever), we will have Quartz Composers plugin instance handle loading our test app. and we have a proxy window class that handles virtual input from our Quartz Composer plugin. The ofxQCBaseWindowProxy is a simple ofAppBaseWindow class I made that allows your test app to be see a “window” even tough you don’t have access to one in Quartz Composer land (you are only handed off an OpenGL context by the parent app loading QC, no window pointer or anything of the sort). We handle mouse (and eventually keyboard) input to our “window” from Quartz Composer.

Once our new OF powered plugin inits we simply create a new ofxQCBaseWindowProxy, and call our normal ofBaseApp initialization stuff:

		windowProxy = new ofxQCBaseWindowProxy();
		ofSetupOpenGL(windowProxy, 1024, 768, OF_WINDOW); 
		ofRunApp(new testApp());

		// we have to manually run setup()
		ofGetAppPtr()->setup();

The dimensions of the window set up the assumed coordinate space of the Open Frameworks calls you make in draw(). Quartz Composer uses a different coordinate space (-1 to 1 for width and -aspect ratio to aspect ratio for height). This lets us draw like we would using GLUT and not have to re-code all of our assumptions. It also lets us use getWidth() etc and have it return sane results.

Now we have to handle a draw loop and event loop, but really Quartz Composer has methods called every frame, so we just use that and manually call setup (via our window proxy) once right after init, and call update and draw using our window proxy within Quartz Composers executeAtTime: method. We expose mouse input via regular Quartz Composer plugin input ports, and can pass in any data we want (real mouse or virtual) for mouse coordinates and button states. Keyboard handling is not up and running yet, but is possible in a similar way.

The basic executeAtTime method looks something like:

	CGLContextObj cgl_ctx = [context CGLContextObj];
	CGLSetCurrentContext(cgl_ctx);
	CGLLockContext(cgl_ctx);
	
	// handle virtual input to our of testApp Pointer
	if([self didValueForInputKeyChange:@"inputMousePositionX"] || [self didValueForInputKeyChange:@"inputMousePositionY"])
	{
		float normalizedMouseX = (self.inputMousePositionX + 1 * 0.5);
		float normalizedMouseY = (self.inputMousePositionY + 1 * 0.5);
		
		float mouseX = normalizedMouseX * windowProxy->getWindowSize().x;
		float mouseY = (windowProxy->getWindowSize().y) - (normalizedMouseY * windowProxy->getWindowSize().y);
		
		ofGetAppPtr()->mouseMoved(mouseX, mouseY);
		ofGetAppPtr()->mouseX = mouseX;
		ofGetAppPtr()->mouseY = mouseY;
	}
		
	// do our actual per frame drawing here.
	windowProxy->update();

	windowProxy->draw();
		
	CGLUnlockContext(cgl_ctx);

Really thats it for the basics. The only gotchas are OpenGL in OF not liking CGLmacros so you loose some speed by manually calling CGLSetContext. The only other things that need tweaking are the data path for loading resources, ( easily done via ofSetDataPathRoot() ), testing addons, and making sure each OF QCPlugin we make has a unique plugin identifier (because you know we will have more than one… ).

Now, things get a bit more complicated if you want to pass in images to Open Frameworks from QC. This is totally doable, but can’t be done easily, so ill eventually supply an ofxQCImage class that will allow a for an ofImage to be initted from a pre-existing texture. Passing images out of QC should be no problem as there are built in provisions for making QCImages out of existing textures (which you can get out of an ofImage).

I’ll try and provide 3 example projects shortly that demonstrate how to use Open Frameworks to create a simple Consumer plugin (rendering directly to the OpenGL world), a simple Provider plugin (typically outputting an image for Quartz Composer to be able to use elsewhere), and a simple Processor plugin which takes in an (input image and outputs an image – think video effect).

Just to get an idea of how awesome this is, Here is the OF QC Plugin running within VDMX. No extra work was needed to be done. You get all of this inter-application capability simply by being usable within Quartz Composer. This means After Effects compatibility with 3rd party wrappers, iChat, PhotoBooth, Final Cut, the list goes on.

VDMX + OF (via QC… lots of acronyms!)

And being post processed by VDMX with my blurs:

vade [at] vade [dot] info