Video Breakdown: GM's Unicorn, Cruise, Shows Off Level 4 Skills in SF—Passing Uber & (Maybe) Waymo

General Motors acquired Cruise Automation in March 2016 for a reported $1 billion (well, at least $581 million). GM President Dan Ammann made a point of being in the press release photograph with Cruise founders Kyle Vogt and Daniel Kan (see photo below). On January 19, 2017, Vogt posted on Twitter: "Took GM Pres. Dan Ammann for a ride in a Cruise AV."

We imagine that Ammann was suitably impressed with the ride in the self-driving Chevrolet Bolt around Downtown San Francisco, as the video posted by Cruise Automation of this ride shows off some self-driving skills that we have not seen before from any other developer.

Self-Driving on the Streets of San Francisco Is Really Hard

In 2015, a very senior autonomous vehicle developer told me that we are closing in on NHTSA Level 4 capability (where the vehicle is capable of unmanned operation), and that within 5 to 10 years we'll see these vehicles driving safely around San Francisco.

This person explained to me that driving the roads in Mountain View and Sunnyvale is one thing, but in San Fransisco, it is a whole different ball game and a much harder problem to solve. He pointed out that you can expect to encounter cars doing strange maneuvers, pedestrians spilling across the curbs, cyclists weaving in their lanes, etc. So, when we see the streets of San Fransisco conquered, then we know that self-driving is ready to come of age.

This would imply that we shouldn't be seeing a video like this Cruise one, demonstrating this level of self-driving skill, for at least another three years. This left us very intrigued.

Analysis of the Cruise Automation Self-Driving Video

We analyzed the sped up video above and pulled out all of the important skills that were being used by the self-driving Bolt. The time references below are for the time displayed in the video, not the video's time.

• 11:10 - A bus is at a stop and is partially blocking the right lane. The Cruise vehicle stops, checks that there are no other road users nearby on the left, and then starts to pull across to the left in order to safely pass the stationary bus. But the bus then decides to pull away, and so the Cruise vehicle continues in the same lane behind the bus.

• 12:30 - Coming up to a narrowing of the lane, whilst in heavy traffic, there is a car parking (possibly illegally) on the right side and a bicyclist undertakes the Cruise vehicle. The cruise vehicle waits for the bicyclist to pass safely and holds the lane until it can pull around the parking car. The Cruise vehicle then proceeds, only to have to slow down as another vehicle from the left lane cuts across it into its lane.

• 13:20 - The Cruise vehicle makes a right turn at a complex intersection whilst avoiding pedestrians crossing on the crosswalk, and after a bicyclist has undertaken it on his way straight across the intersection (but switching from the road and onto the marked crosswalk stripes).

• 14:45 - The cruise vehicle appears to stop for a pigeon walking in its lane and proceeds when the pigeon has moved on.

• 16:45 - After crossing a busy signalized intersection, the Cruise vehicle slowly follows the bicyclist that undertook it whilst waiting at the signals. It had already seen the sideways vehicle blocking the lane ahead and it waits for this other vehicle to finish moving across its lane before it then proceeds.

• 17:30 - The Cruise vehicle makes a left turn at busy signalized intersection where it has to yield to oncoming vehicles before it can make the left turn.

• 20:50 - The Cruise vehicle finds a white delivery van with doors open, and a man working out the back of the van. The van is double parked alongside cars at the curb, and there is barely room to pass the van safely without crossing the double yellow center line (which is illegal). The Cruise vehicle waits and passes safely when the way is clear in both directions.

Cruise states in the intro to the video that "It was fully autonomous." Throughout the video, the green autonomous icon stayed highlighted. This seems to indicate that there are no disengagements of the automated driving system throughout the 22 minute ride, which simulates a passenger using the Cruise mobile app being picked up, and then dropped off at their desired location.

The Combination of Cruise's Self Driving Skills Is Remarkable

To the best of our knowledge, no other technology developer has released video of their vehicles driving around complex downtown areas like this—without disengagements of the automated driving system. Invariably, there are safety drivers ready to take over when these disengagements occur, and automated complex city driving this far has seen safety drivers intervening on a fairly regular basis.

The combination of skills demonstrated in the video is pretty remarkable. The most striking being that the vehicle could deal with highly complex intersection situations, where it had to track and understand the movements of many vehicles, bicyclists, and pedestrians, all on their different trajectories. It dealt with obstructed lanes, and having to make a decision to break the law regarding partially crossing a double yellow lines, just as every other road user had to do if they wanted to get past an awkwardly parked van. It even dealt with a pigeon, similar to the squirrel situation that Google indicated that they couldn't handle back in 2014.

Other Self-Driving Developers & Downtown Self-Driving

During Uber's automated Volvo SUV test in December 2016, the automated cars were driving around San Francisco and there were reportedly several incidences of them running red lights and swerving into bicycle lanes. California eventually banned the Uber test and took the cars off the road.

A couple of months earlier, Uber was demonstrating their vehicles in Pittsburgh, and NPR's Liz Reid reported that her Uber driver had to take manual control of the car to drive around a parked car blocking their lane, similar to some of the situations in the Cruise video. The driver informed the reporter that the car "could have navigated that situation, but in the interest of safety, he decided to go manual."

During an October 2012 presentation at the University of Alberta, Sebastian Thrun, who headed Google's Self-Driving Car project, described how during the first couple of years, from 2009 to 2011, the team specifically aimed their guns at situations they thought "the car [would] never be able to do." They scoped out 1,000 miles of the hardest California terrain—including downtown San Francisco.

Thrun famously presented their autonomous vehicle driving on San Francisco's crooked Lombard Street during his TED presentation in March 2011. But he never mentioned how many disengagements occurred on these drives, and at that time, disengagements were much more common. By Q4 2015, the cars drove 785 miles per disengagement, but we don't know how many miles were on highways (a situation universally recognized as being significantly easier).

So, while other self-driving cars have travelled the complex city streets of San Francisco, none have provided evidence, as Cruise have done, that they've managed it without disengagements.

There's Always a 'But'—But It's Still Remarkable

There always seems to be a "but" when evaluating self-driving technology, so in the time-lapsed Cruise video, it is not clear if the vehicle is driving as smoothly as an experienced human driver would. Also, we have no idea if this was just one run that went perfectly, out of hundreds of runs with disengagements. We have been in communication with Vogt and Cruise, but we haven't yet received any answers to our written questions at the time of posting this article.

But the fact that the Cruise vehicle negotiated this route at all is remarkable. The fact that it dealt with such a complex driving task consistently and safely, whilst dealing with ambiguous road situations, is remarkable. The fact that it negotiated the route without any disengagements is even more remarkable.

We haven't seen any other developer of self-driving technology laying claim to being able to drive in the downtown of a major city without experiencing any disengagements. We suspect that Waymo (formerly Google) has this capability, but they haven't provided evidence to that effect that we are aware of.

Yet, here it is, on Cruise's video, demonstrating a capability that was meant to take at least three more years. To achieve this level of skill in autonomous driving, so soon, is a momentous feat.

• Follow Driverless on Facebook and Twitter
• Follow WonderHowTo on Facebook, Twitter, Pinterest, and Google+