Showing posts with label Starship robots. Show all posts
Showing posts with label Starship robots. Show all posts

Thursday, August 27, 2020

Oh, patents! Starship robots (3) Low-light navigation

Copyright © Françoise Herrmann

One of the main advantages of using delivery robots, to solve the last mile logistics of transporting goods, is that robots can operate 24/7 without mandatory resting periods, or extra pay. Last mile robotic deliveries thus already appeared as quite an atractive solution, considering the surge of ecommerce and online deliveries, even before the pandemic,  a forciori during the pandemic, and the gradual re-opening of economies.

However, for robots to moonlight (pun intended) at no extra costs, they also have to be able to navigate in low-light conditions. Easier said, than done.  How does a camera sensor capture the image of an objet that is no longer visible? How can terrain be mapped accurately and efficiently at night? The StarshipTechnologies patent WO2019086465A1, titled Visual localization and mapping in low light conditions, precisely adresses this issue.

 The patent discloses a SLAM (Simultaneous Localization and Mapping) method where the robot can estimate its own position on a map, while simultaneously creating the map, enabling it to continue its route autonomously. The method relies on data collected by a wide variety of sensors, such as GPS, Lidar, gyroscope,  cameras,  odometer, accelerometer and magnetometer. At sundown, when the sun is astronomically positioned between 0 and 6 degrees below the horizon, a twilight map is created. The twighlight map has the advantage of having both daytime features (e.g. ; straight lines) and night time features (e.g.,urban lights) features.  Thus, the twilight map, in fact, bridges the visibility gap by mapping the position of nighttime features onto a daytime model. In turn, position relative to visibility is triangulated with data incoming from other sensors, and mapping is adjusted accordingly. Otherwise captured images might also be downsized to bring blurry or jagged lines into sharper focus, during mapping.  Likewise roads and buildings might be tagged relative to daytime and night time features to facilitate localization.

The abstract of the invention is included below, together with the patent Figure 4 showing a twighlight map with night time visual features (e.g., urban lights) 2T, and day time visual features (e.g., lines) T1,  extracted during twighlight time, when the sun was positionned astronomically  between 3 and 8 degrees below the horizon.

The present invention relates to a method comprising generating a map comprising day-time features and night-time features, wherein the position of night-time features relative to the day-time features is determined by at least one image captured during twilight. The present invention also relates to a corresponding processing unit configured to execute such a method. [Abstract WO2019086465A1]

Most of the time, all goes well. The 99% autonomous Starship robots fullfill their missions, delivering goods at extended hours, seven days a week, to happy customers. For example, according to the Youtube video incuded below, Starship Robots fulfilled 2500 deliveries during their first week of operation at the Univesrity of Houston, TX, in 2019. However, on occasion the robots get stuck. The following Youtube video shows how a Starship robot was rescued by a University of Houston student, in the middle of the night. Equipped with voiced interaction routines, the Starship robot even gratefully thanked the student, after being rescued.  


Reference
Starship Technologies

Sunday, August 23, 2020

Oh, patents! Starship robots (2) The mothership

Copyright © Françoise Herrmann

 The Mercedez-Benz robovan has been called the Starship robot mothership (Vincent, 2016). An adapted Mercedez-Benz Sprinter van, the Mercedez-Benz robovan ferries eight Starship robots for 99% autonomous parcel delivery, directly to clients, on an algorithm-optimized route. Eight Starship robots enter the van from the rear, using a ramp, and exit the van from the side, also using a ramp. The Starship robots deliver goods within a 2-mile radius. The goods, stored in robovan bins above the robots, are manually loaded into the robot’s payload compartment. After delivery of the goods, directly to customers, the Starship robots return back autonomously to the mothership, where they can dock to recharge, and be refilled with a new payload. The robovan mothership, together with its fleet of eight Starship robots, makes an estimated 400 deliveries per 9-hour day, which solves the last-mile logistics of delivering goods, efficiently and cost-effectively (Burgess, 2017). Especially in response to the surge in demand for online and contactless deliveries, during and post, pandemic.

 The YouTube video below shows the Mercedez-Benz robovan, together with the Starship delivery robots.


 The robovan mothership invention is recited in a family of three patents.  

The patents reciting the Starship mothership vehicle invention are surprisingly wider in scope than the Mercedez-Benz robovan embodiment. Looking at the British patent, for example, the definition of the term “vehicle” is extended to:

 “a passenger car, rail vehicle, watercraft (e.g. ship), underwater vehicle or aircraft.” [0006]

 Likewise, the definition of the term “delivery robot” has a much wider definition, understood in particular to mean: 

a self-driving delivery robot, a self-flying delivery robot (drone), a self-controlling floating vehicle, etc..” [0007]

Otherwise, the delivery robots are described as autonomous vehicles, able to charge or refuel autonomously inside the mothership hold. The robots are also described as preferably fully autonomous. Thus, the robots are equipped with navigation and positioning means, as well as a robot-guidance system, a 2D/3D guidance route, and sensors to collect recordings of the environment, for evaluation in regards to existing obstacles. Equipment ultimately designed to enable the delivery robots to locate a customer’s address and to return to the mothership, without the assistance of a human operator.   

The mothership vehicle is equipped with automated mechanical clamping means to secure the robots in place within the vehicle’s hold. Such automated mechanical clamping means are also described as partially inflatable, in order to accommodate different design contours or angles of the individual delivery robots, loaded into the vehicle hold. 

Communication between the vehicle and the robots is Bluetooth® enabled. For example, communication to monitor loading and unloading of the robots, to transmit the recipients’ delivery addresses to the robots and instructions for remitting the payloads to the recipients, to launch automated routines, as well to transmit information to a communication center and/or to the vehicle driver, for monitoring and oversing the condition and activity of both the mothership and the robots.


The mothership vehicle is also equipped with accumulators, able to interface with each individual delivery robot, for the purposes of charging or refueling, particularly during vehicle travel. Advantageously, charging might be designed contactless via induction. In any event, charging occurs without human intervention. 

To optimize the vehicle’s automated handling of the delivery robots, the cargo hold is also equipped with sensors, able to communicate information in regards to the number of robots in the hold and their position. Likewise, the vehicle is also equipped with means to record and perform the loading and unloading of the robots into the vehicle hold, whether loading and unloading invoke ramps, platforms, and/or a docking interface. 

The British patent abstract of the Starship vehicle invention is included below, together with the patent Figure drawings 1a & 1b of the mothership vehicle, loaded with delivery robots. In Figure 1a, the vehicle (1) is further depicted with a non-inflated fixing device (12) for securing the robots (50) in the hold (10). In Figure 1b the fixing device (12) is shown inflated, and actively securing the delivery robots in place, in the cargo hold (10). 


The invention relates to a vehicle (1) for accommodating a number n ≤ N of delivery robots (50) in a cargo compartment (10) of the vehicle (1), where N is the maximum number of delivery robots (50) which can be accommodated in the cargo compartment (10) and n is the number of delivery robots (50) currently in the cargo compartment (10). The vehicle (1) has the following: - a fixing device (12) for the automatic individual fixing of N delivery robots (50) in the cargo compartment (10), - a communication interface (14) for communication between the vehicle (1) and the n delivery robots (50), and - a number N of charging interfaces (16) for the individual automatic charging of energy stores of the n delivery robots (50) in the cargo compartment (10) [Abstract GB2573382A].

____________________

Notes 

(1) A Patent Cooperation Treaty [PCT], United Nations World Intellectual Property Organization [WIPO] patent, filed in German, by StarshipTechnologies.
(2) A British patent, filed in English, by Starship Technologies
(3) A German patent, filed in German, by Daimler AG

References 

Burgess, M. (Sept. 7, 2016) Mercedes vans filled with swarming delivery bots could be heading to your hometown – Wired Mag.  https://www.wired.co.uk/article/mercedes-starship-drones-delivery-van

Daimler.com (Jan. 13, 2017)  Mercedes-Benz invests in Starship Technologies, the world's leading manufacturer of delivery robots.  https://media.daimler.com/marsMediaSite/en/instance/ko/Mercedes-Benz-Vans-invests-in-Starship-Technologies-the-worlds-leading-manufacturer-of-delivery-robots.xhtml?oid=15274799

Starship Technologies - https://www.starship.xyz/

Vincent, J. (Sept. 6, 2016) Mercedes Benz has made a ‘mothership’ van for six-wheeled delivery robots. The Verge. https://www.theverge.com/2016/9/7/12830298/delivery-bot-van-mercedes-starship-technologies

Monday, August 17, 2020

Oh, patents! Starship Robots (1)

 Copyright © Françoise Herrmann

Brought to you by the Finnish founders and designers of SKYPE, Starship robots are small 99% autonomous vehicle robots, able to operate within a 4-mile radius. These elegant little robots, equipped with sophisticated obstacle-avoidance technology, Lidar sensors, cameras and voiced interaction capacity, roll around sidewalks on 6 wheels, at pedestrian speed. They are 99% autonomous because their routes are always monitored by human remote operators, just in case something went wrong.

Starship robots deliver goods, food and packages, usually in less than 15 minutes, directly to ordering customers in those areas where vendors have contracted the fleets. This way, for example, at a corporate campus, no one wastes time waiting in line, or having to go to a restaurant. The robot meets the customer, upon demand, at a pinned location to deliver orders, sending notifications of estimated arrival time and arrival, via the Starship app, which also handles orders, payments, route tracking, and unlocking of the robot to access the payload. Alternatively, Starship robots also offer a sustainable ecommerce delivery option that decongests roads, where too many delivery vehicles are circulating to meet the ever increasing demands of online customers. In other words, in solving the “last mile” logistics of moving goods efficiently, and cost-effectively, the little electric vehicle robots also do their part to reduce global warming.

The YouTube video below shows an example of a Starship robot delivery. 


Prepandemic, the Starship fleet of 150 robots had driven more than 9000 miles, in 53 cities, in 16 countries, where they met with more than 1.2 million people. In the US, Starship also partnered with DoorDash, specialists in delivery services. 

During the pandemic, the demand for delivery robots such as the Starship robots skyrocketed, according to Forbes.com contributor, Bernard Marr, on May 29, 2020.  Social distancing orders and the lockdown were the primary reasons for the exponential increase in demand, since the driverless robots (which never get sick) can deliver contactless service (e.g.; groceries, take-out food, prescriptions, or small parcels etc.) directly to clients, sheltering-at-home. Considering the convenience, Marr even predicted that Starship robots, and other similar delivery robots, would be here to stay as part of the new-normal post-pandemic. Likewise, The New York Times correspondent Cade Metz and business journalist Erin Griffith, reached the same conclusions, when they stated: “The sudden usefulness of the robots to people staying in their homes is a tantalizing hint of what the machines could one day accomplish — at least under ideal conditions.“ The only caveat will be obtaining municipal clearance for the increased number of vehicle robots circulating everywhere on sidewalks and streets.

All aspects the Starship robots and delivery system are patented. From insulation capacity of the payload container, to the robotics of obstacle avoidance, edge finding and navigation in low-light conditions, including systems and methods of freight delivery and distribution.  Approximately 100 patents have been granted to cover all the inventive aspects and components of the Starship Robots.  

The following is an exemplary and non-exhaustive list of the Starship patents, including World Intellectual Property Oganisation (WO- Patent Cooperation Treaty) patents, United States (US) utility patents, Canadian (CA) patents, British (GB) patents and European (EP - European Patent Convention) patents:

  • US10005609B1 Device and system for insulating items during delivery by a mobile robot
  • CA187741S Delivery robot
  • CA187743S Delivery robot
  • CA187742S Delivery robot
  • CA174441S Delivery robot
  • CA167756S Delivery robot 
  • CA172068S Delivery robot
  • GB2567988A System and mobile freight station, and method for distributing, delivering and collecting freight
  • US20190168392A1 Storage system, use and method with robotic parcel retrieval and loading 
  • US20180349834A1 Method and system for delivering items
  • US20180253108A1 Mobile robot system and method for generating map data using straight lines
  • US10239378B2 Robot and method for traversing vertical obstacles
  • US10239378B2 Robot and method for traversing vertical obstacles
  • US20180244327A1 Obstacle traversing mobile robot
  • US10282995B2  Mobile robot having collision avoidance system for crossing a road from a pedestrian pathway
  • US9741010B1 System and methods for securely delivering packages to different delivery recipients with a single vehicle
  • US20180232839A1 Method and system for autonomous or semi-autonomous delivery
  • EP3330908A1 System and method for securely delivering packages to different delivery recipients with a single vehicle
  • EP3510562A1 Method and system for calibrating multiple cameras
  • WO2018215579A1 Method and system for swapping and/or charging a battery of a mobile robot
  • WO2018108832A9 Robot, system and method detecting and/or responding to transitions in height

  • WO2018215581A1 A battery and a system for swapping and/or charging a battery of a mobile robot

  • WO2019048332A1

    Mobile robot having collision avoidance system for crossing a road from a pedestrian pathway
  • WO2019053162A1 System and method for item delivery by a mobile robot
  • WO2018206514A1

    A signaling device and system for increasing visibility of a mobile robot
  • WO2019020407A1 Device and system for secure package delivery by mobile robot


  • WO2019068634A1  

    Device and system for consumable item delivery by mobile robot
  • US20190236741A1  System and mobile freight station and method for distribution, delivery and collection of freight

  • WO2018215562A1

     Device and method for detection and localization of vehicles
  • WO2018024851A1 Vehicle

  • WO2018099930A1 System and method for securely delivering packages to different delivery recipients with a single vehicle

  • WO2018215583A1 A device and system for increasing tolerance in a battery station

  • WO2018024852A1 Vehicle having a loading device

  • WO2019086465A1 Visual localization and mapping in low light conditions
  • WO2018077619A1 Sidewalk and edge finder system and method

 References

Starship Technologies (website) https://www.starship.xyz/

Marr, B. (May 209, 2020) Demand For These Autonomous Delivery Robots Is Skyrocketing During This Pandemic. Forbes.com https://www.forbes.com/sites/bernardmarr/2020/05/29/demand-for-these-autonomous-delivery-robots-is-skyrocketing-during-this-pandemic/#7bb9d9fd7f3c

Metz, C. & E. Griffith (May 20, 2020) A City Locks Down to Fight Coronavirus, but Robots Come and Go. NY Times. https://www.nytimes.com/2020/05/20/technology/delivery-robots-coronavirus-milton-keynes.html