We chose to create a Motion Graph, concept introduced by Lucas Kovar, Michael Gleicher and Frédéric Pighin.[1]
In such a graph, each frame is represented by a sampling of the subject's parameters. (position and rotation of each joint)
The edges are clips of frames, and the nodes are certain strategic frames that can be used to connect two clips.
Each movement recorded represents an edge and two nodes. After the construction of this trivial graph, we have to increase its connectivity.
Let's imagine we want to create a new transition between two frames i and j. We have to evaluate if the pair is a good candidate. For this purpose, we choose k, the length (in frames) of the transition we are going to create. To connect i and j, we have to compare the k-length window of frames beginning at i, and the k-length window of frames ending at j.
We notice that the body pose stays unchanged during rotation on its axis or translation along the floor. We need therefore to find the transformation that minimizes the sum of the distances between each pair of joints during two groups of k frames. This optimization has a closed-form solution that was first found by Lucas Kovar, Michael Gleicher and Frédéric Pighin. Once we found the transformation, we compute the distance and decide whether or not to keep it, according to its value. Finally, we compute a new transition by interpolating the root positions linearly and applying spherical linear interpolation on joints rotations.
We had some specific constraints due to the fact that our project treats dance movements.
These constraints made us construct a measure synchronous graph. Indeed, this project is no study on movement itself and as such, but tries to explore the creativity of a real live dancer reacting to a virtual one, and the interactions between the two of them. It appears then interesting to keep a certain rythm so that the two dancer can have a creative dialogue, and the most adaptated is of course music. Musics are divided into measures, which organize them according to the melody. That is why we decide to preserve the size of the melodic sentences during the composition, and we try to build a measure-synchronous graph. This means each motion clip is cut into n-length smaller motion clips, n being the number of frames contained in a music measure. This allows greater connectivity while preserving the rythm constraint, so that the real dancer can still interact with the virtual one on music.
Besides, we had to store additional informations on the graph in order to be able to interact in real-time with the performer. Among these informations, we hold the displacement of each move - for that will help us keep a safe distance between the two dancers -, but also other kinds of informations that would enable us to make the virtual dancer respond in a coherent manner to the movements of the performer, such as speed, acceleration, head orientation...
Once this adaptated Motion Graph is created, improvizing dance is just walking randomly on the graph. We thought however, that we could create different kinds of responses from the virtual dancer according to the situation. For instance, we thought interesting to let possible an interaction between the program and a choreographer, that we will explain in the user interface section.
Live performance of a dancer along with an animation generated in real-time by the program from a motion graph of basic moves.
[1] Lucas Kovar, Michael Gleicher and Frédéric Pighin, Motion Graphs, in SIGGRAPH’02 : Proceedings of the 29th annual conference on Computer graphics and interactive techniques, New York, NY, USA, 2002.