Biosignals are often hard to deal with since the collected data are often hard to interpret, redundant, and noisy. Our approach illustrated in the video is to consider muscle activity as behaving as a spring-mass system. In other words, the muscle is oscillating and its oscillations are damped. For example a smooth repetitive movement will make the muscle oscillating at a certain frequency. On the contrary, jerky movements involve high damping.
The system has been developed by Baptiste Caramiaux, post-doc at Goldsmiths, University of London. A similar system has been previously used in movement-based analysis at IRCAM Centre Pompidou in collaboration with Sarah Fdili Alaoui, Marcos Serrano, Frédéric Bevilacqua:
- S. F. Alaoui, B. Caramiaux, M. Serrano, and F. Bevilacqua. Movement qualities as interaction modality. Proceedings of the Designing Interactive Systems Conference (DIS). pp.761-769, 2012
Some details about the algorithm can be found in:
- B. Caramiaux. Motion Modeling for Expressive Interaction. Proceedings of the International Workshop on Movement and Computing (MOCO). pp.76-81, 2014
The system is used in the performance Corpus Nil by Marco Donnarumma (see dedicated post: http://mgm.goldsmithsdigital.com/2015/05/18/sketches-for-body-machine-configuration-corpus-nil)
Playing the violin
The idea of the experiment is to play the violin with smooth repetitive arm’s movements while transient motion will make that the violin does not sound properly.
In terms of modeling, the muscle is assimilated to a harmonic oscillatory system. Then we perform system identification. In other words, the method used identifies the parameters of the system (namely frequency, damping coefficient and offset) in realtime. These parameters are used to control the violin’s parameters (namely bow pressure and bow speed).
The method for motion dynamic extraction has been implemented as a C++ library and interfaced in Max/MSP and Pure Data. The sensor used is the XTH Sense developed by colleague Marco Donnarumma. The sound synthesis is Modalys from Ircam Centre Pompidou.