For a company that owes a large part of its success to inventions, or doing “crazy” things that become “normal”, according to former CEO Jeff Bezos (2021), perhaps that one day, in the near future, Amazon will get their smart mirror for Virtual Try-On (VTON) perfect enough, so that this invention might also become part of the everyday experience of shopping for clothes, at Amazon. In other words, perhaps that the Amazon VTON smart mirror invention might one day become a “yawn-worthy” invention, as Bezos (2021) further puts it for all the Amazon inventions that “got it right”.
The Amazon VTON smart mirror invention is recited in the US utility patent, US10621785, titled Blended reality systems and methods. A patented invention, just like the now- mainstream Amazon inventions, such as: “customer reviews, 1-Click, personalized recommendations, Prime’s insanely-fast shipping, Just Walk Out shopping, the Climate Pledge, Kindle [and] Alexa”, to name just a few, that Bezos (2021) listed in his farewell email to Amazon employees, on February 2, 2021. Hopefully also, and perhaps most importantly, an Amazon VTON smart mirror invention, in the future of online retail fashion, that might soon work by design, rather than of necessity, arising from further pandemic lockdown orders, or curfews.
Till then, the Amazon VTON smart mirror invention intends to make shopping online a lot easier, resolving issues such as garment fitting, and associated customer satisfaction. The patent discloses a shopping experience where the user can try on clothes virtually. The patent discloses a blended reality where systems and methods are used to blend both projected images (e.g., clothes or background scenery), and captured reflection (e.g.; the user or real objects in front of the mirror), into a single scene, so that users are able to see images of themselves, for example wearing a different outfit, in a real or different (projected) background. The blending of both projected images and captured reflection, without ghosting, is achieved by controlling the amounts of light transmitted to the user through the mirror, and reflected by the mirror to the user, resulting in the controlled superimposition of both projected image and reflection of the user. Thus, the means and methods of blending reality might selectively illuminate or darken portions of the user (see Figure 1B below), while selectively projecting an image on the darkened parts of the user, so that the user in front of the mirror sees a reflection of him- or herself, wearing a different item of clothing (e.g.; a shirt), corresponding to the superimposed reflection of the projected image (see Figure 1C below).
More specifically, the means and methods are those that fuel much artificial intelligence (AI) research in the domain of visual synthesizing, already variously applied in a host of different areas, ranging from fashion to architecture or engineering, and Deep Fakes (Lewis, et.al., 2021; Neuberger, et.al., 2020; Palermo, 2020; Sangkloy, et.al., 2016). Efforts in AI visual synthesizing that have precisely tackled the issues of mapping one high-resolution image onto another, in view of obtaining the photorealistic effects of a blended reality.
In the domain of fashion design, for example, AI visual synthesizer programs allow designers to see how a new outfit might fit on one body or another, posing one way or another (e.g., using 3D Body Labs scans [1], [2]). Likewise, both image and sound synthesizing technologies are together invoked in the generation of Deep Fake videos, where fake images superimpose the real ones, together with fake speech, using the vocal characteristics of the speaker’s voice. Deep Fake research relying on visual synthesizers, whether it is for the purposes of developing countermeasures to control the generation of such materials, indistinguishable from the real ones (e.g., using video authentication signatures)—or not (Palermo, 2020).
In any event, such AI synthesizing technologies have already made waves, making it possible to virtually try on eyeglasses (e.g., Zenni), makeup (e.g., L'Oréal Makeup VTON), haircolor (e.g.; L'Oréal Hair VTON) or hairstyles (e.g., L'Oréal style hair VTON), for example, via the generation of photorealistic images, involving the blending of virtual objects onto captured representations of the user.
The apparatus generates a blended reality view by combining the image projections from the display device 110, behind the mirror 120, with control on the illumination of the user 105 and surroundings, in front of the device, using projectors 130 (Fig 1B). The blended view 116 includes the combination of reflected light 122 of the user, and transmitted light, of the beach scene 112 or shirt 117 (Fig. 1C) projected by the display device 110. 
Systems and methods are provided for generating a blended reality view to a user, the blended reality view combining images reflected by a mirror with images transmitted from a screen behind the mirror. Systems for generating blended reality views can include a display device with a screen positioned behind a mirror. The display device can generate a pattern of illumination and non-illumination on the screen so that the illuminated portions of the screen substantially transmit through the mirror. Projectors can be used to illuminate objects in front of the mirror so that the illuminated objects are reflected by the mirror. In combination, the portions of the screen transmitted through the mirror and the illuminated objects reflected by the mirror can provide a blended reality view to a user viewing the mirror. [Abstract US10621785]
References
Bezos, J. (Feb. 2, 2021) Email from Jeff Bezos to employees. https://www.aboutamazon.com/news/company-news/email-from-jeff-bezos-to-employees
Boyle, A (Jan. 2, 2018) Amazon’s blended-reality mirror shows you wearing virtual clothes in virtual locales.Geekwire. https://www.geekwire.com/2018/amazon-patents-blended-reality-mirror-shows-wearing-virtual-clothes-virtual-locales/
Lewis, K. M. et. al. (Jan. 6, 2021) VOGUE: Try-On by StyleGAN Interpolation Optimization https://vogue-try-on.github.io/
Neuberger, A. et. Al. (2020) Image-based virtual tryon network from unpaired data. Amazon Lab126 Paper presented at The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. https://assets.amazon.science/1a/2b/7a4dd8264ce19a959559da799aff/scipub-1281.pdf
Ong, T. (Jan. 3, 2018) Amazon patents a mirror that dresses you in virtual clothes. The Verge. https://www.theverge.com/circuitbreaker/2018/1/3/16844300/amazon-patent-mirror-virtual-clothes-fashion
L'Oréal Makeup Virtual Try-On (VTON) https://www.lorealparisusa.com/virtual-try-on/makeup.aspx
L'Oréal Haircolor Virtual Try-On (VTON) https://www.lorealparisusa.com/virtual-try-on/hair.aspx
L'Oréal Style my Hair Virtual Try-On (VTON) https://www.lorealprofessionnel.co.uk/hair-looks/style-my-hair
Palermo, J. (Feb. 12, 2020) TEDX Deepfakes: Why you can't believe everything you see. Engineer at Dessa. https://youtu.be/JXBBalHl_cI
Sangkloy, P. et. al. (2016) Scribbler: Controlling Deep Image Synthesis with Sketch and Color. Georgia Institue of Technology. http://scribbler.eye.gatech.edu/paper.pdf
Wiggers, K. (June 5, 2020) Amazon’s new AI technique lets users virtually try on outfits. https://venturebeat.com/2020/06/05/amazons-new-ai-technique-lets-users-virtually-try-on-outfits/
No comments:
Post a Comment