Saturday, November 18, 2017

Oh, patents! Nao's joints

Copyright © Françoise Herrmann

Still a few more state of the art robotics patents! 

Nao’s joints arise as a safety issue in the larger focus on how to prevent unanticipated movement of a humanoid robot, and in particular, injury to a human who might get pinched in the robot’s movements or joints. The anti-pinching system disclosed in WO2015185671, titled Anti-jamming system in a humanoid-type robot, was thus designed to address problems of the prior art focused on solving issues of safety when using humanoid robots, and in particular the risks of pinching.

Indeed, the state of the art had previously explored reducing, or leaving a gap in the movement of a robot’s joints to prevent pinching, which by the same token reduced the sorts of movements that could be performed, and thus limited the anthropomorphism of the humanoid. The state of the art also attempted to limit the force deployed to actuate a joint, but this solution tended to reduce the overall capacity of the humanoid robot, for example, to lift heavy equipment.

In response, this anti-jamming invention thus discloses a passive solution that limits the pinching force of a humanoid joint. It consists in a strategically positioned, soft and flexible membrane, at any area of a humanoid joint (e.g.; hip, elbow, shoulder, knee, wrist) where a risk of pinching might be anticipated. This membrane is able to distort a certain distance (i.e.; the diameter of the round black circle (20) representing a finger in the two included patent drawings), in cases where such a finger might get caught in the humanoid joint, or area between the robot’s trunk and joint.  Thus, while the outer shell of the robot’s limbs and trunk are made of hard plastic, certain researched areas of the robot’s outer shell comprise the contiguous invention areas, made of soft and flexible silicone, or rubber, able to accommodate a finger, that could get caught, and thereby passively limiting the pinching force.

This anti-jamming invention is disclosed in a family of patents. The hyperlinked list of patents in this family appears below. The English abstract of the WIPO member of this patent family, WO2015185671, titled Anti-jamming system in a humanoid-type robot, is also included with two patent figure drawings. The two drawings respectively show a black circle (20) representing a finger potentially pinched in Nao’s elbow joint (below), and a black circle (20) representing a finger potentially caught in the space between Nao’s shoulder joint and trunk (above), with both drawings showing the anti-jamming flexible area designed to accommodates the finger, or to limit the pinching force.

The invention relates to the safety of use of a humanoid-type robot. The humanoid-type robot comprises two elements (2, 3) and an articulation (11) with at least one degree of freedom that connects the two elements, the two elements each comprising a skin (22, 23) delimiting the outer surface thereof, the articulation (11) allowing movement in a given range, a first of the two elements (2, 3) being intended to come substantially into contact with a region (25, 26) of the skin (22, 23) of a second of the two elements at one end of the range. According to the invention, the region (25, 26) is flexible such that it can be deformed by a given distance with a force less than a given force. The first element (3) is coupled to the second element (2) by passing through the flexible region (26).  [Abstract WO2015185671]
Patent Family 
  • WO2015185671 (A1) ― 2015-12-10 - Anti-jamming system in a humanoid-type robot
  • AU2015270477 (A1) ― 2016-12-01 - Anti-jamming system in a humanoid-type robot 
  • CA2950652 (A1) ― 2015-12-10 - Anti-jamming system in a humanoid-type robot 
  • EP3152013 (A1) ― 2017-04-12 - Anti-jamming system in a humanoid-type robot  
  • FR3021573 (A1) ― 2015-12-04 - Système anti coincement dans un robot a caractère humanoide  
  • JP2017516672 (a) ― 2017-06-22 -ヒューマノイド型ロボットにおける噛み込み防止システム
  • KR20170021828 (A) ― 2017-02-28 - Anti-jamming system in a humanoid-type robot
  • MX2016015823 (A) ― 2017-06-28 - Anti-jamming system in a humanoid-type robot 
  • SG11201609419R (A) ― 2016-12-29 - Anti-jamming system in a humanoid-type robot
  • US2017080582 (A1) ― 2017-03-23 - Anti-jamming system in a humanoid-type robot

Thursday, November 9, 2017

Oh, Netflix! Frank and The Robot (2012)

Copyright © Françoise Herrmann

And, now for a bit of entertainment!  

Frank and The Robot (2012) is the story of a robot, a bit like Romeo's alter ego, who is purchased by Hunter to assist Frank, his aging, sloppy, and forgetful father. After a bit of resistance, and much learning on the part of both master and automated companion,  Frank and the robot team together for one last triumphant heist.  

This is the sweetest science-fiction comedy-drama, staring Susan Sarandon and Frank Langella, directed by Jake Schreier, and Winner of the Sundance Festival Alfred P. Sloan Feature Film Award.

Here is the trailer.  This film and preview are rated PG-13. 


Reference
Frank and The Robot (2012). Directed by Jake Schreier. Produced by Lance Acord, Sam Brisbee, Jackie Kelman-Brisbee and Gail Nederhoffer. Witten by Christopher ford

Wednesday, November 8, 2017

Oh, patents! Nao’s sibling (Romeo)

Copyright © Françoise Herrmann

Nao’s youngest and largest sibling is Romeo, a humanoid robot designed as a companion for the elderly. Thus, Romeo is being developed with both specific automated physical capacities and artificial intelligence for cognitive assistance and communication with an elderly population. The robot’s cognitive assistance module includes, for example, breakfast, lunch, nap and dinner routines, and reminders to drink fluids. The conversation module, for example, enables patients to query the robot for time of day, date and the latest news, or for certain tasks, such as “turn on the lights”, “show me a movie”, “answer the phone”, “bring me my glasses”, “follow me”, “let’s exercise:”, “hold this”, “empty the dishwasher”, plus much more, since this is a connected robot, whose functionalities are, in principle, indefinitely expandable with new apps, new extensions, and upgrades. 

Indeed, the Romeo platform was actually tested with a consortium of 16 top industrial and academic robotics research partners, such as the CNRS (France’s National Center for Scientific Research) and INRIA (France’s Institute for Research in Computer Science and Automation), for more than 4 years. This mega-collaboration has given rise to many patented inventions, some of which inform the design of  Nao second-generation robots and the design of Pepper, or alternatively, some of which were tested on Nao before becoming available on Romeo. Such R&D research projects have, for example, focused on hand and wrist operation, gestures to enhance communication, enhanced visual recognition or coordination of visual recognition with gestures in communication, robot safety, and collision avoidance. The following video shows just one such R&D project, focused on Romeo pouring a glass of water, while still tethered to a computer workbench.


The below-listed patent family, related to safe use of a humanoid robot, discloses one of the important inventions, connected to the Romeo project. This is an invention that addresses the issue of preventing damage to surroundings, in case the robot falls, and loses control of its movements. In fact, this invention, titled Safety of a humanoid-type robot in US2017072560discloses an emergency stop button. 

The emergency stop button is triggered when a certain force is exerted that exceeds a given threshold, for example, in case of impact, or if the robot falls, for one reason or another. The button can also be actuated by an operator who wants to take the robot out of service because of some observed malfunction. Otherwise, the force exerted that exceeds a given threshold is preferably exerted by the robot’s movements, and in particular, the robot's head on its trunk, as shown in the appended patent Figure 4, below.

The following is a list of the patents belonging to the patent family disclosing the emergency stop button for the purpose of safely using humanoid robots. The abstract for the US member of the patent family is also included below with Figure 4 of the patent showing the robot's head actuating the switch on the robot's trunk. 
A humanoid-type robot comprises two elements and an articulation with at least one degree of freedom linking the two elements, the articulation allowing a travel in a given range in operational operation, a first of the two elements being intended to come into contact with an abutment belonging to a second of the two elements at the end of the range. According to the invention, the robot further comprises at least one switch. The switch is configured to actuate an electrical contact when a force exerted by the first element against the abutment exceeds a given force. [Abstract US2017072560]
  • US2017072560 (A1) ― 2017-03-16 - Safety of a humanoid-type robot
  • AU2015270476 (A1) ― 2016-12-01 - Safety of a humanoid-type robot 
  • CA2950660 (A1) ― 2015-12-10 - Safety of a humanoid-type robot
  • EP3152008 (A1) ― 2017-04-12 - Safety of a humanoid-type robot
  • FR3021572 (A1) ― 2015-12-04 - Sécurité d'un robot à caractère humanoïde
  • JP2017516671 (A) ― 2017-06-22 - ヒューマノイド型ロボットの安全性
  • KR20170021800 (A) ― 2017-02-28 - Safety of a humanoid-type robot
  • MX2016015822 (A) ― 2017-06-28 - Safety of a humanoid-type robot
  • NZ726224 (A) ― 2017-09-29 - Safety of a humanoid-type robot  
  • SG11201609420R (A) ― 2016-12-29 - Safety of a humanoid-type robot
  • WO2015185670 (A1) ― 2015-12-10 - Safety of a humanoid-type robot 
References
Softbank Robotics
Softbank Robotics - Nao
Softbank robotics - Pepper
Sofbank Robotics - Romeo
CNRS
INRIA

Sunday, November 5, 2017

Oh, patents! Nao's siblings (Pepper)

Copyright Françoise Herrmann

The definition of a humanoid robot according to many of the Softbank Robotics patents is the following:
A robot can be qualified as humanoid from the moment … it has certain human appearance attributes: a head, a trunk, two arms, two hands, etc. A humanoid robot may, however, be more or less sophisticated. Its limbs may have a greater or lesser number of articulations. It may control its own balance statically and dynamically and walk on two limbs, possibly in three dimensions, or simply roll over a base. It may pick up signals from the environment (“hear”, “see”, “touch”, “sense”, etc.) and react according to more or less sophisticated behaviors, and interact with other robots or humans, either by speech or by gesture. [Extracted from US20170197311A1] 
Thus, it probably comes as no surprise that Nao is also part of humanoid family that includes a couple of humanoid siblings. Pepper is Nao’s first humanoid sibling. This Softbank Robotics humanoid robot measures 4 ft in height. Pepper’s trunk is humanoid, with two arms, two hands and a head. The lower half of her body is designed as a skirt comprising wheels to ensure mobility. Pepper is equipped with a 3D camera to detect people and their emotions, as well as her own surroundings. The little robot interacts via a tablet and voice. Her hands, fingers and forearms together display 20 degrees of freedom for gestures, specifically designed to enhance communication.  

Pepper was designed as a business companion, in particular for retail and services. She is the ideal host, fielding reception of clients in hotel lobbies, at airports, and in buildings. She might also be found in stores, where customers might query her about goods or inventory. Pepper is also connected, using apps that can be downloaded to increase, or customize, functionalities. She was also designed with a community of developers in mind, who would provide her with endless new possibilities of use and functionality.

The following video pitches Pepper to developers:


Pepper embodies numerous patented inventions, some of which are found in all the humanoid robots manufactured by Softbank Robotics. For example, see posts referring to the inventions covering Nao's hands and Nao's skull. However, the following two patent families are each respectively specific to Pepper’s mobility, and her ability to dock independently into a recharging base:

Patent Family I (Pepper’s mobility)
  • US20170144299A1 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller
  • AU2015248711 (A1) ― 2016-11-03 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller  
  • EP2933069 (A1) ― 2015-10-21 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller  
  • CA2946049 (A1) ― 2015-10-22 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller 
  • HK1216406 (A1) ― 2016-11-11 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller 
  • JP2017513726 (A) ― 2017-06-01 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller
  • KR20170030078 (A) ― 2017-03-16 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller
  • SG11201608204Q (A) ― 2016-10-28 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller 
  • WO2015158885 (A2) ― 2015-10-22 - Omnidirectional wheeled humanoid robot based on a linear predictive position and velocity controller 

Patent family II (Pepper’s ability to dock herself into her recharging base)
  • US20170080816A1 Battery charging base and recharging method implementing such a base
  • AU2015270600 (A1) ― 2016-12-01 - Battery charging base and recharging method implementing such a base
  • CA2951060 (A1) ― 2015-12-10 - Battery charging base and recharging method implementing such a base 
  • EP3152080 (A1) ― 2017-04-12 - Battery charging base and recharging method implementing such a base
  • FR3021914 (A1) ― 2015-12-11 - Battery charging base and recharging method implementing such a base 
  • JP2017518195 (A) ― 2017-07-06 - Battery charging base and recharging method implementing such a base 
  • KR20170026441 (A) ― 2017-03-08 - Battery charging base and recharging method implementing such a base 
  • MX2016015828 (A) ― 2017-06-28 - Battery charging base and recharging method implementing such a base
  • SG11201609422X (A) ― 2016-12-29 - Battery charging base and recharging method implementing such a base 
  • WO2015185525 (A1) ― 2015-12-10 - Battery charging base and recharging method implementing such a base
References
Softbank Robotics – Pepper
Wikipedia – Pepper (Robot)

Thursday, November 2, 2017

Oh, patents! Nao’s skull


Copyright © Françoise Herrmann

Lots of fragile on-board electronic circuitry inside a humanoid robot that requires protection from falls!

A family of patents, including the US utility patent application US20170043487A1, titled Shock-absorbing device for a humanoid robot, discloses a Softbank Robotics shock absorbing invention. This shock absorber (skull) is designed, in particular, for fall-protection of the electronic payload, found inside a humanoid robot’s head, such as Nao’s (or Nao's sibling called Pepper). 

The Softbank Robotics shock absorber invention comprises specially designed shock-absorbing rubber material, and two additional layers of protection: a deformable outer layer and a hard shell containing the fragile electronics. The shock absorbing skull further comprises an outlet for a fan housed inside the humanoid's head, the fan functioning to cool the onboard electronics. The shock absorbing skull also comprises conduits for channeling sound and light emitted by the onboard electronics through the eye orbits, and the ear openings of the humanoid robot's head. Finally, a microphone, attached to the hard shell, is also insulated with the shock absorbing material.   

The shock absorbing system is disclosed for the front and back of the humanoid robot's head. It is also used on the front and back of the humanoid's torso. 

The sorts of falls that are protected by the shock-absorbing invention, disclosed in US20170043487A1, are shown at the end of the following video. 



The family of patents for this shock absorber invention is listed below:
  • AU2015257687 (A1) ― 2016-11-10 - Shock-absorbing device for a humanoid robot
  • CA2948212 (A1) ― 2015-11-12 - Shock-absorbing device for a humanoid robot 
  • EP3140086 (A1) ― 2017-03-15 - Shock-absorbing device for a humanoid robot
  • FR3020847 (A1) ― 2015-11-13 - Shock-absorbing device for a humanoid robot 
  • JP2017514711 (A) ― 2017-06-08 - Shock-absorbing device for a humanoid robot
  • KR20170058332 (A) ― 2017-05-26 - Shock-absorbing device for a humanoid robot
  • MX2016013596 (A) ― 2017-06-29 - Shock-absorbing device for a humanoid robot
  • SG11201608726X (A) ― 2016-12-29 - Shock-absorbing device for a humanoid robot
  • WO2015169894 (A1) ― 2015-11-12 - Shock-absorbing device for a humanoid robot
  • US2017043487 (A1) ― 2017-02-16 - Shock-absorbing device for a humanoid robot 

The abstract of the US member of this patent family is included below, with the patent drawing 2a showing a bird's eye view of Nao’s shock-absorbing skull. In patent drawing 2a,  the skull cap was removed to uncover the insides of the head. The front (11), and back (12), shock absorbing parts are connected transversely. The eye sockets are labeled (14) and (15).
A shock-absorbing device for a humanoid robot, comprises a rigid structure linked to the humanoid robot, a deformable outer shell, and a shock-absorber; the shock-absorber consisting of a flexible cellular structure comprising a set of cells emerging in a main direction, and being secured to the rigid structure at a first end in the main direction, and linked to the deformable outer shell at a second end opposite the first in the main direction. Advantageously, the outer shell is also linked directly to the rigid structure by means of at least one absorbent fixing of silent block type. The invention relates also to a humanoid robot, and in particular the head of a humanoid robot, comprising such a shock-absorbing device.                  [Absract  US20170043487A1]
References
Softbank Robotics