GMS Interface Design Using controlP5

Posted on April 6, 2009 by John C.S. Keston

I spent most of my week long break from teaching continuing development of my Gestural Music Sequencer. I’m not sure if I should call it a sequencer or an arpeggiator. It’s really more like an instrument than either of those. The Gestural Musical Instrument perhaps?

Anyway, it’s far from complete, but I added the ability to toggle sustain on the notes as well a menu to choose from available MIDI device drivers. I decided to use a library for Processing called controlP5 to build the UI controls as shown in the screen grab to the right. All of the controls allow keyboard input, so the application can function while the interface is hidden, only displaying the video.

I’m also planning on adding a function to drop video files into the application to create musical phrases from pre-recorded video pieces. Here’s a section of audio captured from the GMS while attached to the Java Sound Synthesizer Sun Microsystems driver. The default sound for this device is an acoustic piano. You can hear the sustain stop around fifteen seconds in then come back on at the end.

GMS Piano Arpeggio

Spelunking with the GMS

Posted on March 31, 2009 by John C.S. Keston

A have completed a lot of functionality on my Gestural Music Sequencer recently. I added new keyboard controls to change the durations, create dotted notes, increase and decrease BPM, change to one of four preset scales (including a newly added whole tone scale), and toggle between “free mode” and “BPM mode”.

Free mode ignores the BPM and bases the intervals between notes on the mean brightness level of each frame. Since the brightness levels of video can vary dramatically from one environment to another I added a way to dynamically calibrate free mode. While the GMS is in free mode the up and down arrows calibrate the time intervals between notes, whereas, when in BPM mode the up and down arrows adjust the BPM.

Originally the note durations were set with the up and down arrows. Now it’s done with the bottom row of letters on a qwerty keyboard (z,x,c,v,b,n,m) with z being a whole note and m being a sixty-fourth note. All of these durations can be dotted or un-dotted by pressing the period key. This makes it easy to go from slow to very fast phrases instantly.

Here’s a two minute test recording I made to illustrate some of the new functionality. I used the new whole tone scale, and changed the durations with the new keyboard controls. I felt a bit like a cave explorer while making this recording. I had my Petzl headlamp on so I could gesture with my head as if I was looking down a dark cave, while manipulating the keyboard controls with both hands. I’d include a photo, but that’d be embarrassing.

GMS Spelunking

GMS with New Timing Code

Posted on March 30, 2009 by John C.S. Keston

As I mentioned earlier I’ve added new timing code to the GMS. The feature allows you to dynamically change the duration of notes based on more precise timing. The durations range from whole notes down to sixty-fourth notes. I included dotted notes in the list, but I left out the tuplets for now. Currently changing the durations is achieved by the up and down arrows, but I’ll probably change it so that you can go from a one duration to another without having to scroll through them all. I also setup a “free” mode where the timing ignores the BPM and bases the time intervals between notes on the average brightness of the frame. I’ll write more about that later. For now here’s a test recording using the new timing code at 120 BPM.

GMS with New Timing

Bug in GMS Makes High Speed Glitches

Posted on March 29, 2009 by John C.S. Keston

I’ve started working again on my Gestural Music Sequencer after putting it down at end of the quarter for grading and paperwork. The main task that I’m working on is managing the timing so that it’s no longer tied to the frame rate. To do this I’m using Java’s Thread class in Processing.org to drive the tempo independently from the frame rate. Thanks to toxi over at PostSpectacular.com and his response to a question on precise timing, I learned what was necessary to set note durations based on tempo created from Java’s System.nanoTime() method. So far I have enabled eleven durations, from a whole note down to a sixty-fourth note.

While implementing this feature I inadvertently created a bug that set the time interval to zero. I quickly fixed the bug, but not before hearing some pretty amazing sounding glitches out of Reason as it received a stream – make that a tsunami – of note on data as fast as the processor could send it. Here’s how it sounds when Reason is flooded with note on information. The high pitch ringing is caused by the frequency of notes being sent, which turns out to be about 739 Hertz, in other words, seven hundred thirty-nine notes a second. Not even Ingvay Malmsteen can play that fast.

GMS: Super Fast Notes

Video of the Gestural Music Sequencer

Posted on March 19, 2009 by John C.S. Keston

Here’s a video of me playing with my Gestural Music Sequencer. I’ll upload a better version at some point, but I think you can a least get a sense of how you might use this kind of tool (I’m talking about the sequencer, not the performer). As you can see the video has been mirrored so it’s easier to follow your own movements.

To reiterate how the sequencer works, the X axis of the brightest pixel is used to determine the pitch, while the Y axis is the dynamics. The application outputs MIDI data that I’m routing to Reason. I’ve programmed the up and down arrows on the keyboard to increase or decrease a multiplier that along with the 15 fps frame rate determines the time between each note on event. I’ve also enabled a group of keys to adjust the transposition.