Freshmen URD Track; Entrepreneurship and Innovation
Despite its ever increasing popularity, the guitar is an instrument that is exceedingly hard to play for an amateur. This is generally due to the fact that in order to play a song using the guitar the guitarist is required to know the sheet notes, the relevant chord positions and strum according to the rhythm simultaneously. The goal of our project is to remove this memorization aspect of playing the guitar.
In order to eliminate the need to juggle their vision between 2 possible targets we’ve decided to overlay the sheet notes onto the frets using an AR headset. These cues generated by the headset will tell where on the fretboard the user should press at any given moment allowing them to circumvent the memorization aspect of playing the guitar.
Project Objectives
The end goal of our project is to produce a prototype consisting of a hololens app that is capable of projecting cues onto a guitar’s fretboard and communicating with an external peripheral to convey the rhythm to the user via haptic feedback.
Technologies
Vuforia SDK
Vuforia is a widely used platform for AR development, with support for the majority of phones, tablets, and headgear.
Unity
Unity is an engine used mainly to create games. Unity has many packages that the user can download and use for their own creation, they could also buy game assets from the unity asset store like model and texture.
Hololens
Hololens is a headset developed by Microsoft to be used for augmented reality or AR.
Arduino
Arduino is an open-source electronics platform based on easy-to-use hardware and software. Arduino is able to be modified to fit the user's needs and wants. Using the many components as well as many software users is able to add functionality that the user wants.
Marker tracking Vs Natural tracking
There are many ways to track an object or image. Marker based tracking,as the name suggests, is the usage of markers in order to track an object. This is where natural tracking shines as they are relatively more flexible in the terms of tracking and still accurate enough to be serviceable.
Results
Although we have yet to complete a prototype we have so far made a unity program capable of accurately tracking the guitar and projecting cues onto it. We’ve also integrated a vibration motor into unity so that it will provide haptic feedback when a command is sent to it.
Summary of Accomplishments
We’ve currently managed to successfully track the guitar and project cues onto it alongside integrating a vibration motor into unity to give haptic feedback on the rhythm.
Future Directions
We plan on reworking and finishing up a full prototype by the end of the next semester. This would at its minimum include integrating our headset onto the hololens and adding a UI for song selection. If time allows we also plan on adding additional functionalities to the system such as a lesson component that teaches the user basic guitar chords. We’re also planning on making a companion app for translating and mapping songs of the user’s choice into the system via the use of AI.
