The Microsoft Hands-Free Music Program is a participatory design project intended to extend access to music performance and composition, to musicians stricken with ALS (amyotrophic lateral sclerosis, also called Lou Gehrig’s or Charcot’s disease), spinal chord injuries or other sorts of paralysis conditions. Thus, the project has developed a suite of three eye-controlled applications called The Microsoft Hands Free (Red-Eye) Sound Studio. These applications require no mouse, no keyboard, or other touchscreen interaction, to operate.
The Microsoft Hands Free (Red Eye) Sound Studio Suite includes: Hands-free Sound Jam, Hands-free Sound Machine, and Hands-free Expressive Pixels.
- Hands-Free Sound Jam is an electronic eye-controlled music environment for loop-based music performance and composition. The program offers a “clip launcher”, small musical fragments that can be edited or written, and then strung or looped together into a composition.
- Hands-Free Sound Machine is a 16-step sound sequencer that supports the output of musical compositions to musical instruments, and stage effects.
- Hands-Free Expressive pixels enables authoring and rendering of static and animated LED matrix displays. Used together with Sound Machine, Expressive Pixels augments the musical composition output with visual effects.
Hands-free selection of virtual objects, or the use of eye control to interact with the computer, is an AR (Augmented Reality) invention disclosed in US9201578, titled Gaze swipe selection. The system comprises a connected head-mounted display device (HMD), such as a pair of glasses, equipped to track eye or gaze movements, for example using two or more infrared position sensitive detectors (IR PSD) to track glint position. The eye tracking information is then processed and translated into visual pointer information that has an impact on the virtual objects of the software.
The abstract of this invention is included below, together with the patent drawing 3A, showing an embodiment of a portion of the HMD (Head Mounted Device) that detects user eye movements.
The abstract of this invention is included below, together with the patent drawing 3A, showing an embodiment of a portion of the HMD (Head Mounted Device) that detects user eye movements.
Methods for enabling hands-free selection of virtual objects are described. In some embodiments, a gaze swipe gesture may be used to select a virtual object. The gaze swipe gesture may involve an end user of a head-mounted display device (HMD) performing head movements that are tracked by the HMD to detect whether a virtual pointer controlled by the end user has swiped across two or more edges of the virtual object. In some cases, the gaze swipe gesture may comprise the end user using their head movements to move the virtual pointer through two edges of the virtual object while the end user gazes at the virtual object. In response to detecting the gaze swipe gesture, the HMD may determine a second virtual object to be displayed on the HMD based on a speed of the gaze swipe gesture and a size of the virtual object.[Abstract US9201578]
References
Microsoft Hands-Free Music
Microsoft Hands-Free Sound Machine
Microsoft Hands-Free Sound Jam
Microsoft Hands-Free Expressive Pixels
https://www.microsoft.com/en-us/research/project/microsoft-expressive-pixels/
Music in our heads : From ALS adversity to inspired album – Sharon H. Chnag
No comments:
Post a Comment