Category Archives: GMS

The GMS (Gestural Music Sequencer), written in Processing.org by Unearthed Music recording artist and AudioCookbook.org founder John Keston, analyzes video input and converts it into a sequence of musical information in real-time. Each frame is processed to find the brightest pixel. The x coordinate is converted into pitch while the y coordinate is translated into dynamics. The application was also designed to use probability distributions on pitch and durations.

GMS with New Timing Code

Posted on March 30, 2009 by John CS Keston

As I mentioned earlier I’ve added new timing code to the GMS. The feature allows you to dynamically change the duration of notes based on more precise timing. The durations range from whole notes down to sixty-fourth notes. I included dotted notes in the list, but I left out the tuplets for now. Currently changing the durations is achieved by the up and down arrows, but I’ll probably change it so that you can go from a one duration to another without having to scroll through them all. I also setup a “free” mode where the timing ignores the BPM and bases the time intervals between notes on the average brightness of the frame. I’ll write more about that later. For now here’s a test recording using the new timing code at 120 BPM.

GMS with New Timing

Bug in GMS Makes High Speed Glitches

Posted on March 29, 2009 by John CS Keston

I’ve started working again on my Gestural Music Sequencer after putting it down at end of the quarter for grading and paperwork. The main task that I’m working on is managing the timing so that it’s no longer tied to the frame rate. To do this I’m using Java’s Thread class in Processing.org to drive the tempo independently from the frame rate. Thanks to toxi over at PostSpectacular.com and his response to a question on precise timing, I learned what was necessary to set note durations based on tempo created from Java’s System.nanoTime() method. So far I have enabled eleven durations, from a whole note down to a sixty-fourth note.

While implementing this feature I inadvertently created a bug that set the time interval to zero. I quickly fixed the bug, but not before hearing some pretty amazing sounding glitches out of Reason as it received a stream – make that a tsunami – of note on data as fast as the processor could send it. Here’s how it sounds when Reason is flooded with note on information. The high pitch ringing is caused by the frequency of notes being sent, which turns out to be about 739 Hertz, in other words, seven hundred thirty-nine notes a second. Not even Ingvay Malmsteen can play that fast.

GMS: Super Fast Notes

Video of the Gestural Music Sequencer

Posted on March 19, 2009 by John CS Keston

Here’s a video of me playing with my Gestural Music Sequencer. I’ll upload a better version at some point, but I think you can a least get a sense of how you might use this kind of tool (I’m talking about the sequencer, not the performer). As you can see the video has been mirrored so it’s easier to follow your own movements.

To reiterate how the sequencer works, the X axis of the brightest pixel is used to determine the pitch, while the Y axis is the dynamics. The application outputs MIDI data that I’m routing to Reason. I’ve programmed the up and down arrows on the keyboard to increase or decrease a multiplier that along with the 15 fps frame rate determines the time between each note on event. I’ve also enabled a group of keys to adjust the transposition.

Keyboard Controls to Adjust Transposition in GMS

Posted on March 18, 2009 by John CS Keston

I have recently added keyboard controls to dynamically set the transposition in my Gestural Music Sequencer. I set it up like a piano keyboard, so that “Q” is the key of C, “2” is C sharp and so on. In this example I have the application set to to play a minor pentatonic scale. This makes it quite clear when the key changes. In this case I’m modulating between C and F sharp. I also started with a slower note duration, then switched to a faster one. I would not classify this as a very musical performance. This was part of a test to make sure that all of the transposition key controls were operating properly.

Transposition Controls (C to F# Modulation)

Minor Pentatonic Pattern in GMS

Posted on March 17, 2009 by John CS Keston

As I mentioned before I have GMS setup to produce specific scales. At this stage they are all based on the key of C. Eventually I’ll set up the application so that the key and scale are dynamically adjustable. I will also include a wider variety of scales, including all the modes, diminished, whole tone, and more.

One thing I haven’t decided on how to approach is timing and tempo. The way it works at the moment is that the tempo is determined by applying a multiplier to the frame rate. In this example I’m dynamically changing the multiplier using the arrow keys to achieve different note durations.

GMS Pentatonic Scale