This is a first look at the application that I’ve been working on for the Leap Motion Controller. It provides multi-dimensional MIDI expression through hand gestures.
Geco has been designed for live performance and it operates at extremely low latency, while requiring very little resources on your computer. It can thus perfectly run alongside any MIDI capable software.
In this demo I show some of the configuration capabilities while using Native Instrument’s Razor synth in Reaktor.
40 different control streams with both hands
any control stream can be mapped to MIDI CC and Pitchbend messages on 16 different channels
instantly switch between related control streams by opening or closing your hands
carefully designed GUI for an immediate overview of the active MIDI mappings
real-time low-latency visual feedback of your hand movements and MIDI data
integrated virtual MIDI port on MacOSX
connects to any known MIDI output port on your computer
fully customisable user interface (colours, graphical elements)
flexible document management that can be loaded while performing gestures
high performance and near-zero latency engine with virtually no CPU impact when the real-time visualisations are hidden
MIDI decimation setting to allow integration with legacy hardware that has limited MIDI bandwidth
Yesterday afternoon I took a few hours to write an EigenD agent for the Leap. Currently it only supports sending out the x, y and z axis of two palms relative to the Leap device itself. The API is very intuitive and it shouldn’t take long to gradually add support for hand directions and fingers, I wanted to first play around with it for a while though.
In this experiment I control two effects in Native Instruments’ Guitar Rig 5. It’s processing one of my songs that’s playing from iTunes and you can hear the recorded result. I did have to manually align the audio to the video, so that might not be perfect.
This developer version of the Leap device supports a little bit less range and a restricted field of view, which is why it’s sometimes missing some detection at natural boundaries. The production versions will not suffer from this.
This video quickly demonstrates some new capabilities of the ThingM blink(1) USB RGB light agent that I wrote for EigenD. You can now send commands that turn individual light channels on for a certain period of time. This can then be used to sequence them live using the Eigenharp’s arranger functionality.
I resumed PC/Mac gaming when friends suggested that I’d try out Guild Wars 2 and play with them. Since I’ve been mostly a console gamer for the past years, I didn’t feel comfortable using my keyboard to play games, it’s just not as immersive.
I tried other alternatives like the PS3 Dualshock, the SpaceNavigator 3D mouse, the Alphagrip iGrip keyboard, iPad gestures, … and a Logitech G13 keypad with a Logitech G600 MMO gaming mouse was the only thing that felt usable, but still not perfect.
That made me think, I had these awesomely sensitive Eigenharp instruments with 3D control over every key, very light pressure, natural hand positions and more than enough keys. It suddenly made sense that they would work just as well for gaming as for music. I thus created an EigenD extension agent that allows the mapping of any Eigenharp key to any keyboard key, as well as mouse movements and mouse button clicks. All of this can of course be further configured with different thresholds, sensitivity settings, multi-axis velocity-based triggering and much more. Currently this agent is only available for MacOSX, but it shouldn’t be hard to add Windows support. Those interested can find the source code on GitHub.
Here’s a short demo video of me playing Guild Wars 2 with my Eigenharp Alpha:
As you can see in this video, the Eigenharp is indeed a wonderful game controller. While I don’t expect people to buy the Eigenharp just for that, existing Eigenharpers now have one of the best game controllers available, all the time while training muscle memory for their musical instrument.
For the curious, this is the legend to the Guild Wars 2 key map I’m using with my Eigenharp Alpha.
Note this is isn’t limited to the Eigenharp Alpha, I initially built a prototype that uses the Eigenharp Pico, lying flat on the table. It works just as well though there are fewer keys and I personally prefer holding my hands in a vertical position to reduce the stress on my wrists.
I was offered a great position in a great company, not in the music sector though. I’ll give more details shortly.
As of next month I’ll be available for work.
Eigenlabs is moving towards a non-profit charity foundation and I’ll only sporadically work as a freelancer for them when the need arises. I’ve been asked if I would be interested to help the new foundation as a trustee and I’m considering to do so. Its goals are dear to my heart and I’ve been very passionate about them for a few years. Since any work I do for The Eigen Foundation in that capacity can’t be remunerated, I obviously need to find another income.
If you have an interesting project for which you think I’m a right fit, don’t hesitate to contact me!
I received my blink(1) USB lights a few days ago and last night while waiting for our guests to arrive, I quickly coded an EigenD agent for them. You can connect up to 16 blink(1) lights and control them all through EigenD and your Eigenharp. This can for instance be connected to the metronome, or scheduled with talkers on the arranger or even connect your keys’ expression directly to the colors.
A first composition with Animoog 2 and my Eigenharp Alpha. I’m using a single sound patch in Animoog that totally changes based on how I touch the keys on the Eigenharp. The sound ranges from something akin to a polyphonic theremin to percussive glass sparkles as well as everything in between. This patch was based on one of Animoog’s Acoustic Expansion Pack patches.
Animoog 2 has improved its MIDI handling tremendously and together with an iConnectMIDI interface, I’m able to send the Eigenharp expression data to Animoog at full rate without any decimation. Creating and playing music with Animoog is now a very emotional experience. Currently this only reacts to polyphonic aftertouch, but I’m hopeful that Moog might ultimately add full support for Voice-Per-Channel MIDI, maybe even at 14-bit resolution.
Animoog sounds amazing to me and is amongst the top software synths I’ve played.
Eigenharp Alphas are built in batches when enough people have committed to buy one.
Eigenlabs has put a page online where you can register your interest, it costs nothing to do so and does not commit you to purchase. When there are enough people signed up, Eigenlabs will contact you for a deposit and they will make your instrument.
I received an early copy of Audiobus for iOS yesterday to try it out while playing live MIDI through it. I was pleasantly surprised that it barely adds any latency and reacts really well. The shortcut bar is extremely useful and app-specific icons makes it very easy to interact with backgrounded applications while having another app in the foreground.
In this short demo I’m using my Eigenharp Alpha with a pre-release of PPG WaveGenerator. It’s fed into JamUp for additional effects and amp simulation. Loopy HD sits at the output side to build up the loops. I’m also using FunkBox, MIDI-synced to Loopy, for the drums.
The Eigenharp is connected with iConnectMIDI through a USB hub, into the camera connection kit to send the MIDI with virtually no latency and at a very high bandwidth. I use a Matrix USB/SPDIF interface that’s also plugged into the USB hub and recorded the iPad audio digitally without any post-processing.