As a part of their Spring 2016 capstone project, electrical engineering students, Yedi Luo, Qing Ran, Huy Nguyen, Yuhao Wang and Jiwei Wang, created a robotic hand that plays the piano through both autonomous and semi-autonomous modes.
The modes are two-prong. The robotic hand can play autonomously by reading sheet music via Optical Character Recognition. This computer vision technique transmits play commands between the computer and the robotic arm.
This hand can also play semi-autonomously. A user can place a glove with sensors on his or her arm. As the user moves, the robotic hand will mimic his or her movements.
For two quarters, the students tweaked and tested the devices under the mentorship of Professor Howard Chizeck. In the end, what they achieved was a delicate dance between technology and art – something quite beautiful.
