MetaGestureMusic draws upon three complementary research fields to create gestural, interactive musical instruments for both musicians and non-musicians alike. It draws upon Auditory Culture studies to understand the cultural significance of music. It uses User Centered Design (UCD) methods to involve the end user in scenario building and creation of design mock ups. These ideas will then be implemented in functional interactive musical instruments that are built and programmed using techniques from the field of New Instruments for Musical Expression (NIME). It deploys machine learning techniques to decode gesture captured by sensor technologies such as physiological biosignal consumer accelerometers on iPhones, motion capture systems and GPS to correlate to features of novel digital sound synthesis. This creates forms of embodied musical interaction, engagement with music that is physical, situated, social, and participatory. This vision of music opens up the creative process of music and embraces all acts of engagement with music, from selecting, to listening, to dancing, to performing.

The¬†MetaGestureMusic project has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP/2007-2013) / ERC Grant Agreement n. FP7-283771.


The use of technology to build musical instruments is not a new idea. Since the advent of electricity many inventors have used technology to create instruments which allow musicians to perform with electronic sound. A famous example of this is the Theremin, an instrument created in 1919 which allowed musicians to manipulate the pitch and volume of an oscillation using  arm gestures. Today the Theremin still has a place in popular culture and is found in film scores, popular and experimental music. What process of selection has allowed has allowed one instrument from this era to gain such notable recognition?Recently a whole new series of digital instruments has emerged. These range from the Tenori-on, developed by well known musician Toshio Iwai and commercialized by Yamaha, to Guitar Hero, a guitar like input device for a play-along game. Such instruments are all innovative, and successful in a range of ways, however for each one that succeeds ten more fall into obscurity. Do they then contribute to a modern day tradition of the creation of new digital musical instruments? Are there design principles which could allow for the creation of instruments with a broader impact? Are there development methods which can engage with the end user and allow us to learn what is important for them in musical expressivity?

The contention of the MetaGestureMusic project is that by understanding the cultural importance of music and coupling that with innovative ways of interacting with sound, we can create profound embodied musical experiences that will transcend the traditional roles of the music listener and the music maker.


Objective 1: To bring a critical view to interactive music systems, informed by cultural studies approaches
This will allow us to conceive new forms and formats of music that use digital and interactive technologies but do not depend on classical or popular music tradition or style, instead drawing upon our understanding of the cultural role music plays in all human activity.

Objective 2: To build Expressive Musical Instruments which Afford Embodied Musical Experience
We will build working systems that implement the principles of embodied interaction and deploy functional prototypes of interactive instruments on widely available mobile devices to be used and tested by a broad user base.

Objective 3: To establish a new, democratic definition for music as a fundamental human activity
This project will provide the cultural understanding and develop the technologies necessary to carry out this ambitious vision. Cultural insight and user-centered approaches will ensure a relevance to everyday users of the technologies we build. These technologies, in turn, will facilitate the potential uptake of this vision in society by providing the means to demonstrate through example and practice-based dissemination and share knowledge and transfer the skills inherited in this way of engaging with music.