Waymo Applies Google Image Recognition to Autonomous Vehicles

can be reached at meehna@gmail.com
can be reached at meehna@gmail.com
Share on facebook
Share on twitter
Share on pocket

Waymo, the self-driving technology company, just came out with the modestly named Content Search, but it could have huge implications for advancing autonomous vehicle technology.

  • In collaboration with Google Research, Waymo developed Content Search to catalogue and find relevant images contained in Waymo Drivers’ logs.
  • To drive safely, autonomous vehicles have to be able to recognize and correctly react to objects they could encounter on the road.
  • The more accurate and reliable the information given to an autonomous vehicle, the safer that vehicle becomes.

Waymo Content SearchWaymo’s new Content Search tool allows engineers to catalogue and find billions of images. (Photo: Waymo)

Seeing better

As explained on its blog, Waymo and Google Research, both divisions of parent company Alphabet, collaborated to create Content Search. By leveraging the search technology similar to what powers Google Photos and Google Image, Waymo engineers can now quickly locate just about any object stored in Waymo’s driving history and logs through 20 million miles of collecting data on the road. In essence, the Content Search turns all the objects into a searchable catalogue, accurately tracking billions of images.

Waymo Drivers use machine learning to detect and classify different types of objects encountered on a road. Using powerful neural nets, Waymo’s perception system learns to recognize objects and their behaviors from already labeled examples.

In order to safely operate, an autonomous vehicle must be able to distinguish between a snowman or a person in the middle of the road. It also has to recognize the difference between a vehicle and a cyclist, while figuring out if an object is a pedestrian or someone on a scooter. Waymo already demonstrated its technology can recognize and avoid a cyclist.

Waymo Content SearchBy applying Google’s image search technology to driving logs, Waymo developed Content Search to quickly find and catalogue objects. (Photo: Waymo)

Relevant results

Here’s where the power of Content Search ups the game. Instead of having to painstakingly cull through images, Content Search can quickly pull up relevant images that engineers can feed into the Waymo Driver AI systems to learn. This “knowledge transfer” between sister companies allowed the application of the technology Google uses to find all the dogs in your Google Photos album, to finding all the dogs a Waymo Driver drove past in the logs.

Content Search offers the flexibility to search sensor logs via a “similarity search” and hone in on objects in ultra-fine-grained categories as well. In addition, it can search by text in the image, such as on street signs. With the ability to accelerate labeling, Waymo has improved its Waymo Drivers intelligence to detect such things as school buses with children or a cat and dog crossing a street.

As one Waymo vehicle gets “smarter with time and experience,” the rest of the fleet benefits by sharing the information. Through this collective and efficient learning system, Waymo says it self-driving cars will be able to better adapt to changing environments.


The better machine learning gets, the safer autonomous vehicles will become, and the sooner we will see true autonomous vehicles on the road.

About the Author

can be reached at meehna@gmail.com
Close Menu

We use cookies and browser activity to improve your experience, personalize content and ads, and analyze how our sites are used. For more information on how we collect and use this information, please review our Privacy Policy. California consumers may exercise their CCPA rights here.