Gestural Music Interface in Processing

Posted on March 14, 2009 by John CS Keston

A big thanks goes out to Jason Striegel and Nick Watts for inviting us to perform at Make: Day at the Science Museum of Minnesota. I performed with my group Keston and Westdal. Other performers included Savage Aural Hotbed and Tim Kaiser. Besides the performances there were some excellent presenters. Myself, Nils Westdal, our drummer Graham O’Brien, and our intern, Ben Siegel greeted visitors at our table. We presented bits and pieces that Graham used with his drums including sticks, pencils, and a chain. We also showed materials from Unearthed Music, Audio Cookbook, and I revealed a gestural music sequencer (GMS) I developed in Processing.

I was really excited to see the reaction to the sequencer. The application samples video and displays it inverted so it looks as though you’re looking into a mirror. Each frame is analyzed for brightness, then the X and Y data of the brightest pixel is converted into a note. The X axis is used to select a pitch, while the Y axis determines the dynamics. As visitors moved, danced, or gestured in front of the camera notes were generated based on a predetermined scale. Here’s a short sample of what the GMS can produce. I’ll post more about this soon.

Gestural Music Interface

Processed Noise Extracted from 1972 Dialogue

Posted on March 8, 2009 by John CS Keston

What you hear in this clip is all of the processed noise extracted from 1972 Social Commentary Degraded with a Halftone Pattern pasted together. I ran the lot through a short ping pong delay to give give it some stereo processing. Secondly, I decided to leave this file in its 48kHz 24bit state before rendering the MP3. This way you can tell whether or not you have the latest Flash plugin. I just played it on a computer with an older version of Flash Player 10 and got some horrible crackling in the playback, however, if I play it in QuickTime by clicking the link it sounds fine. If you get the same problem, try upgrading your Flash plugin in the browser to fix it.

Processed Noise from 1972 Dialogue

Audio Playback Bug Fixed in Flash Player 10.0.22.87

Posted on March 8, 2009 by John CS Keston

On December 1, 2008 I posted an article about a playback bug in Adobe Flash Player version 10.0.12.36. At that time I was encoding a majority of my audio clips at 48kHz and 24bit, then compressing the MP3 versions to 192kbps. This caused the audio to playback in the Flash player poorly with considerable amounts of warping and distortion. Since then I have been down-sampling my clips to 44.1kHz and 16bit before creating the MP3 to avoid the issue. Although the fix came out early in February, I think I will continue doing this for the time being, until most readers have had time to upgrade their Flash plugins. If you ever look at archived or random entries, I’d suggested upgrading so you don’t experience the playback problems.

Kutiman Remixes YouTube on Thur-You.com

Posted on March 7, 2009 by John CS Keston

Early last week a student showed me Thru-You.com, a collection of YouTube video mashups by Israeli, neo-funk artist Kutiman. I was thoroughly impressed with his work, but soon afterward the site went down due to bandwidth limits. Since then it’s been up and down several times. Hopefully they’ll have their bandwidth problems sorted out soon, so If you can’t view these videos straight away, bookmark the site and check back because they’re worth the wait. Update: The videos have been moved to YouTube, so there should be no more bandwidth issues.

Captive Mosquitoes make Music in Innovative Installation

Posted on March 2, 2009 by John CS Keston

Ali Momeni has recently posted an article about his and Robin Meier’s installation Strategies for Post-Apocalyptic Computation that was shown at the 2009 Spark Festival. Based on the article it looks as though they have renamed the piece Truce. Here’s an excerpt from the article:

Our installation explores reciprocal musical interactions between the mosquito and the computer. The computer produces a stimulus signal to which the living mosquitoes synchronize. Subsequently, the computer sings a third voice that responds to the musical inflections of the mosquitoes’ buzz. These three voices come in and out of harmony depending on the mosquitoes propensity to maintain its sync with the stimulus signal.

I made three or four visits to this fascinating installation during the festival. Here’s a segment from a recording of the piece that I made during one of my visits. To see some well produced video of the installation please visit the original article.

Brief Recording of Truce