VR multiplayer percussion ensemble with haptic interaction

The VR multiplayer percussion ensemble with haptic interaction project aims to allow two players to each select a percussion instrument (drum or marimba) and meet in a virtual space to perform together. Users can practice individually on each instrument, and if they wish to play in an ensemble, they can enter a virtual ensemble room and select an available instrument (an instrument already selected by the first player is not available). The player who enters the virtual ensemble room first has the authority to select a song for the performance, and once the second player joins, the chosen song can be played together.

Each user is represented by an avatar with an upper-body rig. The avatar’s head is mapped to the HMD, while the avatar’s hands are mapped to the user’s real hands, which are tracked via the Leap Motion camera attached to the HMD. This setup allows users to turn their heads and wave their hands, enabling natural interaction with their virtual ensemble partner. The environment was developed using Unity and Photon 2 (PUN 2).