Sunday, February 7, 2021

Oh, patents! Smart mirrors (1) Mirror.com

Copyright © Françoise Herrmann

Need a workout in the safety of your own home? Pilates, sculpt, post-natal exerices, kickboxing, weight training, ballet, dance cardio, hip-hop, tai-chi, yoga flow, boxing, kettlebell?  Mirror.com offers more than 50 different sorts of workouts, on-demand, from beginning to expert levels, in 5 to 60-minute, private or live, group sessions with certified trainers, or using a library of tapes. Each workout is brought to you directly in your living room, via a smart mirror. With a monthly subscription for up to six different family members, users can tune-in to the (smart) Mirror, both for access to workouts and live feedback. Feedback on such items as posture, position and performance, using monitored heart rate data that the Mirror records via Bluetooth, and data on every user-move, collected via proprietary camera algorithms.

Considering that the coronavirus was found to also spread via airborne transmission, even outdoor exercising might appear risky. In other words, transmission via droplets >5 µm that are expelled via sneezing, coughing, talking or singing loudly, are believed to drop to the ground within about 3 feet, whereas airborne transmission is suspected to occur via suspended droplets <5 µm, containing the virus (Ming Hui Chua, et al., 2020). Smaller particles that stay suspended in the air for longer periods of time, traveling greater distances, depending on wind for example. Thus, the Mirror workouts probably appear as safe as you will ever get. Not to mention the fact that the Mirror workouts might also be more social, to the extent that it is possible to connect with friends, to work out with them, in a live, trainer-led, (Zoom-type) session, in contrast to working out, while maintaining at least 6 feet of distance, in the open air.

Master-minded in 2018 by CEO Brynn Putman, originally a professional ballet dancer with the New York City Ballet, and then founder of the Refine Method Fitness Studio, the Mirror.com start-up was sold to the Canadian athletic apparel retail company LuluLemon Athletica Inc., in the summer of 2020, for 500 million USD  (O'Brian, June 29, 2020, Terlep, S. June 30, 2020; Riley Moffat, Sept. 17, 2020). The concept no doubt appeared potent, since the Mirror not only brings professional boutique fitness right into people’s living rooms, it also brings safety in the midst of state-mandated lockdowns, caused by a virus transmitted largely via the respiratory route.

The (smart) Mirror personal home fitness system is an invention with many different patented aspects, such as the following, all of which might be licensed and used:

  • continuous acquisition of biometric and movement data via camera and sensor, during a workout session
  • reality-blending aspects of 3D video output on a display screen, together with mirroring functions displaying the user’s reflection
  • embodied, fitness expert-system that enables to customize and adjust an exercise program, based on acquired biometric data and user motion, while providing real-time feedback on performance (e.g., calories burned and heart rate.)

The US utility patent, US10845511B2, titled Smart Mirror, filed by Hewlett Packard in 2016, for example, recites a blended reality aspect of smart mirror inventions. In other words, how both the display of data (collected from the user and/or imported) and mirror reflection of the user are processed for presentation to a user, facing the smart mirror. Specifically, the patent recites visualization of both interface data and user reflection, relative to the user’s captured eye focal distance. Eye focal distance is measured, using binocular cameras, as the convergence point of the user’s left and right eye gaze. 

Thus, the invention discloses a highly visible (bright) mirror reflection and less visible (dimmed) interface data, when the user’s captured eye focal distance is on the mirror image. Conversely, when the user’s captured eye focal distance is on the display, then the mirror image presented is less visible (dimmed), while the interface data becomes highly visible (brighter).  Consequently, users are able to use their gaze on specific objects of the user interface, as an input method to control the smart mirror, while the mirror uses biometric information (the eye focal distance of the user’s gaze) to determine which information to present on screen. Finally, hands free operation of the information displayed, in turn, is disclosed as extending the usefulness of the mirror. Although, considering the invention was filed several years prior to the COVID 19 pandemic, it is quite incidental that such a “contactless” feature of the invention would become especially significant.

The abstract of this invention is included below, together with Figures 5A and 5B, showing the smart mirror’s differential response to the captured eye focal distance of a user 505, facing the display .  

Figure 5a illustrates the smart mirror 500, with the user 505 focusing his gaze 520A on his reflection 540A, whereas Figure 5b illustrates the smart mirror’s appearance, with the user  505 focusing his gaze  520B on interface objects 530B of the display. The user’s eye focal distance is measured with a binocular eye sensor 510, located on top of the mirror frame. Measurement is restricted to the user’s reach.

 Small screen areas, 550A on Figure 5BA, and 550B on Figure 5B, respectively show an overlap of the user reflection 540A and the interface objects 530B. However, according to the inventive determination of eye focal distance, the user reflection 540A is more prominently visible in Figure 5A, whereas the interface objects are more prominently visible in the Figure 5B.

In one example, a smart mirror. The smart mirror includes a display adjacent one side of a half-mirror, sensors to acquire biometric information of a user facing an opposite side of the half-mirror, and a controller coupled to the sensors and the display. The controller determines from the biometric information a focal distance of the user. If the focal distance corresponds to the display, the controller presents a user interface on the smart mirror in a highly-visible manner and a reflection of the user in a less-visible manner. If the focal distance corresponds to the reflection of the user, the controller presents the user interface on the smart mirror in a less-visible manner and the reflection of the user in a highly-visible manner.  [Abstract US10845511B2]

The YouTube video below shows the elegant Mirror, in use, for several sorts of living room workouts.

References

Lululemon Athletica Inc. https://info.lululemon.com/about

Ming Hui Chua, et al.(Aug 7, 2020) Face Masks in the New COVID-19 Normal: Materials, Testing, and Perspectives. Research (Wash D C). 2020; 2020: 7286735. Published online 2020 Aug 7. doi: 10.34133/2020/7286735    https://pubmed.ncbi.nlm.nih.gov/32832908/ 

Mirror.com https://www.mirror.co/

Refine Method Fitness Studio  http://www.refinemethod.com/

O’Brian S.A. (June 29, 2020) Lululemon to buy mirror at-home fitness startup for 500 million. CNN.com. https://www.cnn.com/2020/06/29/tech/lululemon-mirror-fitness-startup-acquisition/index.html

Riley Moffat, A. (Sept. 17, 2020) Lululemon’s Mirror Workout Tool Doesn’t Sell Sports Bras -- Yet. https://www.bloomberg.com/news/articles/2020-09-17/lululemon-s-mirror-workout-tool-doesn-t-sell-sports-bras-yet

Terlep, S. (June 30, 2020) Lululemon Buys Mirror, an At-Home Fitness Startup, for $500 Million. WSJ. https://www.wsj.com/articles/lululemon-to-buy-at-home-fitness-company-mirror-for-500-million-11593465981

No comments: