Category Archives: GMS

The GMS (Gestural Music Sequencer), written in Processing.org by Unearthed Music recording artist and AudioCookbook.org founder John Keston, analyzes video input and converts it into a sequence of musical information in real-time. Each frame is processed to find the brightest pixel. The x coordinate is converted into pitch while the y coordinate is translated into dynamics. The application was also designed to use probability distributions on pitch and durations.

GMS Videos and Interface

Posted on by
Reply

Today I made two more videos with the GMS using a different technique than in Dodger of Red Lights. This time I wanted to explore what could be produced using visual input that I wasn’t directly controlling. The obvious, albeit cheesy, lava lamp came to mind, so I turned it on and went to work.

I connected a separate video camera via firewire, leaving a screen with all my interface controls available for me to tweak while recording. I recorded the video to MiniDV on the external camera, and as before recorded the output onto my PCM-D50 to free up the CPU for Reason and the GMS. Since I was recording the video on the external camera I had to the flip the video so that the image corresponded properly with the sound. This led me add a feature for toggling the mirroring in the GMS.

Click the image to see a full sized screen shot of the interface. All three videos are available on the Unearthed Music Vimeo account. Here’s an excerpt of audio from the piece titled Sound Globules.

GMS: Sound Globules

GMS Piece: Dodger of Red Lights

Posted on by
1

Today I decided to see if I could create a version of what I performed at the MAW outing last night, as a streaming video with good quality sound. I opted not to use screen capture software so that all the processing on my Macbook Pro could go into rendering the images and sound. The imagery was created by manipulating a red, LED, bike light near the lens of the camera. The screen was then shot with a second camera while the audio was recorded externally using my PCM-D50.

I built a pentatonic scale using the note probability distribution feature in the GMS. The most likely note was a C followed by D, G, G#, and then B natural. I also used the probability distributions on the durations to get some stumbling rhythms centered around sixteenth notes.

Dodger of Red Lights from Unearthed Music on Vimeo.

GMS Probability Distributions

Posted on by
Reply

The most recent addition I have made to the GMS is probability distributions for note pitches and durations. The way this works is that the user adjusts sliders from zero to one-hundred for each note in the twelve tone (chromatic) scale.

When note probability distributions are enabled, the randomness is weighted so that notes set to zero will never play while at one-hundred the note has the maximum probability of playing. This way the user can setup any sequence of notes in a scale or chord, then toss in a small (or large) possibility for an altered or dissonant note.

The same is true for duration probabilities except that I have the dotted durations as a separate possibility that is applied to the duration determined by the weighted random function. Here’s an example of the note and duration probability distributions in action with the settings as shown in the image.

GMS Probability Distributions

GMS Interface Design Using controlP5

Posted on by
1

I spent most of my week long break from teaching continuing development of my Gestural Music Sequencer. I’m not sure if I should call it a sequencer or an arpeggiator. It’s really more like an instrument than either of those. The Gestural Musical Instrument perhaps?

Anyway, it’s far from complete, but I added the ability to toggle sustain on the notes as well a menu to choose from available MIDI device drivers. I decided to use a library for Processing called controlP5 to build the UI controls as shown in the screen grab to the right. All of the controls allow keyboard input, so the application can function while the interface is hidden, only displaying the video.

I’m also planning on adding a function to drop video files into the application to create musical phrases from pre-recorded video pieces. Here’s a section of audio captured from the GMS while attached to the Java Sound Synthesizer Sun Microsystems driver. The default sound for this device is an acoustic piano. You can hear the sustain stop around fifteen seconds in then come back on at the end.

GMS Piano Arpeggio

 

Spelunking with the GMS

Posted on by
Reply

A have completed a lot of functionality on my Gestural Music Sequencer recently. I added new keyboard controls to change the durations, create dotted notes, increase and decrease BPM, change to one of four preset scales (including a newly added whole tone scale), and toggle between “free mode” and “BPM mode”.

Free mode ignores the BPM and bases the intervals between notes on the mean brightness level of each frame. Since the brightness levels of video can vary dramatically from one environment to another I added a way to dynamically calibrate free mode. While the GMS is in free mode the up and down arrows calibrate the time intervals between notes, whereas, when in BPM mode the up and down arrows adjust the BPM.

Originally the note durations were set with the up and down arrows. Now it’s done with the bottom row of letters on a qwerty keyboard (z,x,c,v,b,n,m) with z being a whole note and m being a sixty-fourth note. All of these durations can be dotted or un-dotted by pressing the period key. This makes it easy to go from slow to very fast phrases instantly.

Here’s a two minute test recording I made to illustrate some of the new functionality. I used the new whole tone scale, and changed the durations with the new keyboard controls. I felt a bit like a cave explorer while making this recording. I had my Petzl headlamp on so I could gesture with my head as if I was looking down a dark cave, while manipulating the keyboard controls with both hands. I’d include a photo, but that’d be embarrassing.

GMS Spelunking