Real-time recurrent image editing modulated by sound amplitude

Recurrent image editing modulated by sound amplitude in real time from sonat on Vimeo.

kinect camera to control stop-motion video flow

In this post I would like to explore embodied ways on how one could interact with a visual stream. More specifically, this is about controlling the flow of a video stream with one own's body movements.

The system I am using is based on Microsoft's Kinect camera. On the computer side I am using the Quartz Composer to connect different streams of information on the one side body related parameters and on the other parameters related to the video stream.

In the following video, I am showing a simple situation where the movie is controlled by vertical movements of my hand.


Kinect controlled video playing from sonat on Vimeo.

Kaleidoscope + Stopmotion + Photographic Content + Minimal Electro

This was a project I was working on since long time. I am really happy that it is now here finally "published". These photos were taken as in a typical stop-motion setup. In Quartz Composer, I connected the speed of the photographic flow to the amplitude of audio signal at a given frequency band (usually low frequency, because beats are located there). This makes the video move faster with each beat. In addition to this, I applied different visual symmetry effects, known as kaleidoscope. The outcome is visually unusual when typical urban or natural scenes are used and fits perfectly to minimal electro. Two performances follow, the first illustrates the concept using photographs taken in an urban area, namely Osnabrück, Germany. And the second one, which uses images taken in a natural park.
 

Real-Time Halftoning with Perfect Circles

How to transform an image in real-time into an array of perfect circles (or other geometrical shapes) with their diameters proportional to the local image brightness below the circle?

This is classically known as halftoning and one of the most ancient digital image manipulation methods, still used today in printing business.

I managed to do this using Quartz Composer, here is an example video. In this video, the resolution i.e. the number of circles used, changes over time. Video starts with black and white circles, continues with circles colored with original pixel values and finishes with random colored circles. It is all done in real time without any noticeable delay.

 

Perfect for VJing purposes. When real-timing is not necessary the content of the video can be a political figure for example...


Halftoning AngelaMerkel from sonat on Vimeo.

The concept can be generalized to other politicians or other half-toning shapes...

 
Halftoning Politicians from sonat on Vimeo.


Halftoning AngelaMerkel Hearts from sonat on Vimeo.


Cross modal integration with auditory modality could be implemented by adding a noise to the diameter of the circles that is proportional to the amplitude of the instantaneous sound signal..