To design and implement an image processing system that takes as input, live video of a user playing a virtual musical instrument and generates appropriate music in real-time.
We have designed, implemented and tested a virtual musical instrument system. Our system is low-cost and the user needs to print just one sheet of paper for every instrument he/she wishes to play. The latency of the process is small, and the sound output is near real-time, though there is still room for improvement. Our accuracy for the hit localization is 80% for typical user behavior.