BuildDecember 2017

What are augmented piano lessons?

Projection-based piano tuition that turns the keyboard itself into the interface and the lesson into a game.

Piano keys against a background of musical notes

Augmented reality presents an opportunity to revolutionize the way we learn how to play instruments. Imagine sitting in front of a piano, not knowing how to play it, and being able to play a song by following notes projected on the instrument. Now we have the ability to take the game mechanics of Rock Band and apply them to real instruments. If you are having trouble picturing augmented lessons, take a look at RocketJump’s animation from 2010.

RocketJump's 2010 animation.

What that has to do with me

During a course on project management for computer information systems, I decided to challenge myself by playing the role of developer and creating a proof of concept for augmented piano lessons. I chose the piano because it is a significant segment of the musical instrument market and the instrument I am most familiar with. To avoid expenses, I decided the proof of concept would be triggered by an image target that would be similar to a piano. Once the image is recognized, the software plays a stream of notes that a user can hit to a familiar tune.