Kaleidoscope + Stopmotion + Photographic Content + Minimal Electro

This was a project I was working on since long time. I am really happy that it is now here finally "published". These photos were taken as in a typical stop-motion setup. In Quartz Composer, I connected the speed of the photographic flow to the amplitude of audio signal at a given frequency band (usually low frequency, because beats are located there). This makes the video move faster with each beat. In addition to this, I applied different visual symmetry effects, known as kaleidoscope. The outcome is visually unusual when typical urban or natural scenes are used and fits perfectly to minimal electro. Two performances follow, the first illustrates the concept using photographs taken in an urban area, namely Osnabrück, Germany. And the second one, which uses images taken in a natural park.
 

Real-Time Halftoning with Perfect Circles

How to transform an image in real-time into an array of perfect circles (or other geometrical shapes) with their diameters proportional to the local image brightness below the circle?

This is classically known as halftoning and one of the most ancient digital image manipulation methods, still used today in printing business.

I managed to do this using Quartz Composer, here is an example video. In this video, the resolution i.e. the number of circles used, changes over time. Video starts with black and white circles, continues with circles colored with original pixel values and finishes with random colored circles. It is all done in real time without any noticeable delay.

 

Perfect for VJing purposes. When real-timing is not necessary the content of the video can be a political figure for example...


Halftoning AngelaMerkel from sonat on Vimeo.

The concept can be generalized to other politicians or other half-toning shapes...

 
Halftoning Politicians from sonat on Vimeo.


Halftoning AngelaMerkel Hearts from sonat on Vimeo.


Cross modal integration with auditory modality could be implemented by adding a noise to the diameter of the circles that is proportional to the amplitude of the instantaneous sound signal.. 

Programming a Weather Station with Quartz Composer

This is a weather station clock programmed in Quartz Composer. There are 6 different parameters that are taken into consideration. The temperature, air pressure, rain drop, humidity, wind speed and direction. Weather information is downloaded from Osnabrueck University Physics Department's web page and an XML parser extracts different parameters. These parameters determine then the background color (reddish for warm and bluesh for cold), diameter of the turning rainbow wheel (high air pressure), global blurriness (humidity), speed of the wheel (average wind speed). Two examples illustrates how the clock behaves under different weather conditions:

Hello Quartz Composer

One of the first things I have programmed with Quartz Composer was this rainbow clock. I currently use this as a screen saver.

 
Clock from sonat on Vimeo.

Varela's View of Cognitive Science..

The most integrative figure of the cognitive science ever made. 

This figure has been published in "The Embodied Mind" (1991) by Varela, Thompson and Rosh.

Different directions depict different subdisciplines of the cognitive science. The distance from center represents the philosophical flavor.  While Constructivists are located around the outer shell of the circle, the center part is occupied by Representationalists. The way these latter conceive the mind, is compatible with the view that the mind i.e. intelligent behaviour is a symbolic representational machine, pretty much like a computer. 

These views have completely different opinions on evolution as well.

Representation of Natural Movies across the Visual Cortex

Below is a video showing the spatio-temporal activity patterns in response to artificial and natural stimuli. These beautiful recordings were realized by Dirk Jancke in his laboratory. We compared the activity patterns evoked by natural movies to those evoked by artificial stimuli (such as for example moving edges) that are typically used in physiological experiments.

We are the first research group recording cortical large-scale activity patterns in response to natural movies using the method of voltage-sensitive dye imaging.

Voltage-sensitive dye imaging during natural and artificial conditions. The first column depicts stimuli as shown during the experiment: drifting square gratings (rows 1 and 2) and natural movies recorded by cats (rows 3 and 4). Colored rectangles indicate the position of receptive fields hand-mapped at each penetration site, symbolized with a color-matching circle in the second column. Evoked optical imaging signals caused by these stimuli are depicted in the second column. The scale bar represents 1 mm across cortex. Note that the color code has different scales across different conditions. The third column depicts the time course of spatially averaged activity. The strength of motion flow field is represented in the last column.

Voltage-sensitive dye recordings of cortical responses to natural stimuli and gratings. (A) Two natural movies (blue and orange boxes) and gratings (gray box) used as stimulation are depicted together with evoked cortical responses. Visual stimuli are shown in upper rows within each box (movie 1 and movie 2). Leftmost image represents an example movie frame covering approximately a visual angle of 30° 3 40°. The scale bar represents 5° of visual angle. White rectangle approximates the local portion that directly stimulated the recorded cortical area. The temporal evolution of the movie within the delineated region is shown in succeeding frames. The second row within each box displays activity during intervals of nonoverlapping 100-ms frames including the prestimulus period. The rightmost image shows the average activity computed over the entire stimulus presentation of 2 s. See top left frame for vascular image of the recorded cortical area (P 5 posterior, L 5 lateral; scale bar represents 1 mm). Color bar indicates activity levels as fractional fluorescence change relative to blank. (B) Time courses of global activity computed as the average across all pixels of a given frame. Shaded gray area symbolizes prestimulus period. Line colors are matched to the boxes shown in A; black 5 grating, blue/red 5 natural conditions. The thickness of lines represents CIs computed by resampling all the pixels that belong to a given frame (P 5 10^5). Right panel: Mean amplitudes of activity; error bars represent the SD.

Few more videos:

Representation of Simple Stimuli across the Visual Cortex

Moving edges, which consist of drifting dark and bright bars, are kind of stimuli that are commonly used in physiological experiments mainly because of their parametrically controllable aspects.

The video below shows the large-scale cortical dynamics at the spatial scale of several millimeters during presentation of such visual stimuli.

Overlaid on the well-known orientation maps (shown at the bottom row), which are evoked by the specific orientation of the stimuli, drifting edges are furthermore represented by propagating waves (top row). The original publication can be found here.

NeuroImage from sonat on Vimeo.

Caption: Multiplexing of space and orientation information. The data presented in Fig. 1 and 2 is presented as a video. Upper video: Propagating activity reconstructed by combining oscillatory SVD components, averaged across several propagation cycles during stimulus presentation (cf. Fig. 3 and Fig. 4a). M = Medial, P = Posterior. Lower video: Propagating waves are shown in combination with tonic SVD components representing the orientation maps. The weight of both components were equalized prior to their combination. Contour lines are drawn at 90th activity percentiles of the tonic components.

Decomposition of evoked cortical responses to gratings of 0.2 c/deg drifting for 2 s at a temporal frequency of 6.25 Hz. (a) Evoked spatio-temporal activity patterns (top rows) and time courses obtained by spatial averages across the images (bottom traces) expressed as fractional change in fluorescence relative to blank condition (ΔF/F). Top left frame shows the vascular image of the recorded right hemisphere, P = posterior, L = lateral; here and in all figures scale bar 1 mm. Leftmost frame in 2nd row depicts the time-averaged orientation map derived by subtracting evoked responses to the vertical grating from horizontal. Green trace = responses to vertical grating, drifting rightwards in visual space; blue trace = horizontal grating, drifting downwards. (b) Top left corner, singular values, gi, ranked in order of their contributions. Components of significant contribution to variance are colored (gray area depicts significance level). The contribution of each single SVD component to single recorded trials (n=35) was computed, their correlations across trials are represented as a matrix. Spatial (ui(x)) and temporal (vi(t)) modes of the SVD components were clustered according to their correlation (red, yellow, and green boxes; curves represent weight of each spatial mode [y-axes] as a function of time [400–1800 ms]).