Introduction

Morning practices for a healthy life.

It was during the daily repetition of a small sequence of morning physical exercises that this project was conceived. Exercises of energizing, breathing, stretching, boxing practices, and martial arts originated a synthesis named MartialmentalEX. In order to become more captivating and engaging, this routine needed a soundtrack and this is how, during the composition of this soundtrack, it seemed very appropriate to leave space for the practitioner, musician or not, to play part on that musical work while he exercises.

This is how I started a long search for instruments to control electronic music available on the market, the feasibility of building new instruments, and which softwares were being used to maximize these resources by creating new interfaces and expanding the possibilities.

The term “Electronic Lutherie” fits well into this perspective in which the musicians create their own interfaces in an attempt to express an individual gesture and their own way to deal with composition and performance. About this aspect, the composer Paul Lansky makes the following comment:

The stages

The MartialmentalEX guide envisages the following stages, respecting the sequence of the exercises:

1. Breathing
2. Boxing
3. Circular movements
4. Invisible drums

Stage 1: Breathing

Description: With eyes closed, the practitioner starts the practice by observing of his own breathing; pulse, constancy, durability, etc. Then, he turns his attention to his heartbeats and tries to coordinate them with the breathing.

Activity Planned: An Electrocardiogram system connected to the Arduíno microchip is connected to the practitioner’s body. The system sends data to the Ableton Live software (via Midi/USB), which codifies the heartbeats into sounds of electronic drums that can be recombined, distorted, manipulated with effects, etc.

Activity Achieved: It was not possible to test the system with a real electrocardiogram machine. The system works perfectly in simulation.

Comments: This idea originated from the collaboration with the computational scientist Manuel Guerra, who already uses an electrocardiogram system connected with Arduíno.

Stage 2: Boxing

Description: In a station specially designed for this project, the practitioner stands in front of a “punching bag”, resistant balls, and several devices for impact training . The practitioner can “play” these “instruments” as he strikes them, thus interacting with the song composed especially for this moment.

Activity Planned: Each part of this station contains a touch sensor (trigger) that is connected to a set of electronic drums (alesis Dm4) with p10 cables. It is also possible to expand the possibilities of combinations of sounds and timbres through the interconnection of the set of electronic drums to the software Ableton Live.

Activity achieved: The interconnection of triggers to a set of electronic drums is a technology already established and available on the market since the 1990s. It is working perfectly and it is good entertainment for visitors to the laboratory. The construction of the station per se is the main challenge of this research and is developing very well.

Comments: The objectives at this stage of the project basically relate to sound and impact. Since the integration of the sensors to the set of electronic drums is a technology that offers a lot of stability, all challenges are turned to the design of a feasible, intelligent structure of easy mobility and especially with the necessary resistance. This is a proven therapeutic stage in this the project.

In terms of music, a song of rhythmic structure was conceived, where JAB (left hand for the right-handed) sounded like the sound of a snare drum and the RIGHT (left hand for the right-handed) sounded like the sound of the floor tom.

For a larger variety of timbres, the expansion of this system into using all the resources of the software Ableton Live is already a reality without the need for the Arduíno microchip, through the use of a Midi/USB cable. Thus, the boxing instruments may behave like a keyboard or may be connected to a sampler, offering infinite possibilities and combinations.

Stage 3: Circular Movements

Description: Standing up, the practitioner makes circular movements, upward and downward, with the upper part of the body, bringing his hands from the top of his head to his feet, touching the ground, rotating the torso, touching the ankles, etc.

Activity Planned: Standing in front of a video camera, the software openFrameworks will map the practitioner's body considering the ends of his body as data source. These data, which range from 0 to 100 and from 0 to -100, are sent to the Ableton Live software and will be connected in a synthesizer and manipulated in terms of pitch.

Activity Achieved: The integration between openFrameworks and Ableton Live has not been fully successful yet (OSC). The research is seeking the best way to link these data and, if necessary, the transposition to the software Processing will be made.

Comments: A feature natural to the evolution of the project is the forms of capturing gestural movements with no touch, and their codification. Here, the project faces a challenge: mapping the practitioner's body and its interaction with the background, given that the sensor used in this stage is the computer’s internal camera.

Here, the range of the research widens and we do not have a relation of connected/disconnected, of inside/outside anymore, such as was the case with the touch system; however, we do have a spatial relation where many of the elements that surround the practitioner may interfere in the sound that is produced as well as his precise position.

Ideally the practitioner is able to move, dance, and control sound at the same time, while the song composed especially for this stage stimulates his movements.

Stage 4: Invisible Drums

Description: The sensation of being enveloped by a sensory bubble was a key element in this research. Being able to strike the invisible, to strike a physically nonexistent instrument... this is the objective of this stage. In an area previously marked out, the practitioner stands in front of the computer video camera, where the system Isadora processes his movements allowing strikes in the air and the production of sound.

Activity Planned: We developed a program within the software Isadora, a system that allows the creation of modules that send commands and parameters to other programs, setting pitch, time, duration, and several specifications of interactivity. It works well with Ableton Live.

Activity Achieved: We have reached the stage where a fast movement with a strike is accurately captured, but we still need to make some adjustments in terms of sound response. Currently this response is slow and undefined (error of latency). Tests with laser sensors are planned in order to improve the effectiveness of the invisible drums.

Comments: It was during the workshop on the software Isadora, with collaborator Fernando Rabelo, that the idea of expanding the physical space for the invisible came up, because the system allowed this long-awaited interaction. This is a very important step in the project because it opens doors for several other forms of interaction and music making in itself, and can even serve as a basis for research with disabled people and the creation of multiple sensory spaces.

Complementary Information:

The Interaction with the Public:

Considering all these possibilities of interactivity, a new conception now emerges in order to complement the live performance and to expand the practice field of the project MartialmentalEX.

If, so far, only the practitioner controlled the parameters of the sound, produced noise and effects, now it is possible to install a system with a sensor directed to the public, which will also be able to alter certain pre-defined aspects of the piece that is being performed on stage.

Thus, the relationship artist/public ends up being questioned and, even better, experienced in practice.

The Exhibition

MartialmentalEX was conceived as a 10- to 15-minute musical performance. After the presentation, the instruments will be available in the space to be experimented by the visitors. The name, previously conceived only as MartialEX, gained another member due to the expansion of sensation and of experience. From tactile to invisible. In a mind connection.

BIBLIOGRAPHY

Lansky, P. (1990). A View from the Bus: When Machines Make Music. Perspectives of New Music, 28(2), 102-110.

ABOUT THE AUTHOR

Fabiano Fonseca is musician, soundesigner and graphic designer from Belo Horizonte.