Plano-Based Toyota Uses 4D Radar to Detect Passengers Who Have Been Left Behind » Dallas Innovates


Heartbreaking headlines become more frequent with the arrival of each summer: a child left in a hot parked car has died of heatstroke. In 2021 alone, 23 American children died of heatstroke after being left in vehicles, according to the nonprofit Kids and Cars.

Plano-based Toyota Connected North America hopes to help end those stories forever with a new patent-pending concept: Cabin Awareness, technology using high-resolution, millimeter-wave 4D imaging radar. Radar helps detect occupants, including some pets, in vehicles and has the potential to detect them if they have been overlooked.

The radar sensor is mounted out of sight above a vehicle’s headliner and can detect the presence of a “life form” in the vehicle, even after the driver has exited.

“Detect micro-movements”

Toyota Connected “Cabin Awareness” concept uses new technology to detect occupants [Diagram: Toyota Connected North America]

The Cabin Awareness concept works by detecting “micro-movements” like a heartbeat, movement and the breathing of occupants across three full rows of seats, the cargo area and the floors.

The technology classifies all occupants based on their height, posture and position, supporting advanced safety applications. Toyota says its platform offers “robust detection” even if the occupant is a baby covered in a blanket. In this scenario, a passerby would not know that a baby is in the car, but Cabin Awareness would have the ability to detect it.

According to Toyota, current technologies such as weight sensors can be prone to false alerts and detection errors, and cameras can be limited by blind spots. While other in-vehicle radar systems currently exist, Toyota claims they have a limited passenger detection range.

The potential to save lives

On average, one in four children who die in a hot car got inside while it was unattended or non-operational. And it’s not just in the summer that this happens: the inside of a car can reach 125°F in minutes, even when the air outside is as low as 60°F.

The body temperature of children increases three to five times faster than that of adults, according to studies. If left in a vehicle during the summer, the interior cabin temperature can reach dangerous levels in just 10 minutes. That’s what makes technology like Cabin Awareness so essential.

“Toyota Connected’s talented software engineers and data scientists harness advanced technology to bring innovation and cutting-edge technologies to customers’ vehicles,” said Zack Hicks, CEO and President, TCNA and EVP and Chief Digital Officer, Toyota Motor North America, in a statement. “We are extremely proud of our efforts to take this idea from the drawing board to a full-fledged concept and hopefully develop technology that has the potential to save lives.”

Testing technology soon in Arlington

Currently, the new technology is officially only a “concept”. But a Toyota partner, May Mobility, is offering it a real-world trial. The autonomous vehicle company is testing Cabin Awareness in its fleet of Toyota Sienna AutonoMaaS (Autonomous Mobility as a Service) minivans at its Michigan headquarters. It will soon begin testing the technology in public AV deployments in Arlington As good as inside Ann Arbor, Michigan. Additional deployments will take place in late 2022.

Recently, May Mobility announced that a portion of its fleet will be modified to become ADA compliant and accommodate wheelchair passengers.

“As we move closer to chauffeured operations next year and continue to grow our global business with more public AV deployments, technology like the Cabin Awareness concept is essential for our passengers,” said May’s CEO. Mobility, Edwin Olson, in the press release.

Sending alerts when a passenger is forgotten

The Cabin Awareness concept currently provides a series of warnings to help alert the driver – and potentially passers-by – to check the rear of the vehicle if a living thing is detected.

First, a light flashes on the dashboard. Then the horn sounds and the hazard warning lights flash. After all these early warnings, the owner can receive a notification on their phone via the Toyota app, as well as text messages.

According to Toyota, Cabin Awareness can even send alerts through smart home devices or send text messages to emergency contacts designated by the primary user. Other alert options include Toyota’s Safety Connect emergency assistance system contacting first responders through the vehicle’s in-vehicle SOS feature.

The development team continues to explore additional notification alerts, including vehicle-to-vehicle (V2V) communications and the incorporation of smart infrastructure detail signage.

Inspired by NASA space technology

The inspiration for the Cabin Awareness concept came from microwave radar technology created by NASA’s Jet Propulsion Laboratory, Toyota says. NASA technology supported underground rescues after 7.8 magnitude earthquake Nepal in 2015. NASA engineers and rescuers were able to detect human breathing and heartbeats under more than 30 feet of rubble, helping responders know where to dig to save them.

“NASA’s use of radar technology was inspiring,” said Brian Kursar, CTO at TCNA and Group Vice President, CTO and Chief Data Officer at TMNA, in the statement. “The idea that you can listen to heartbeats using contactless technology opens up new possibilities to give Toyota the potential to produce a service that is beneficial to the evolution of our in-vehicle services.”

For parents, shuttle operators and more

A future use of the new technology is to notify parents when their children have completed a self-driving shuttle ride. An autonomous shuttle can also delay driving to the next stop if it senses someone is still in the vehicle who shouldn’t be, Toyota says.

“The main difference with this system is the improved resolution and accuracy, full cabin sensing and breadth of functionality provided by Cabin Awareness,” said simon robert, a chief engineer at TCNA who led the development process. “With the accuracy of these sensors, we are designing Cabin Awareness with the goal of reducing false positives and false negatives.”

Born from a hackathon

The idea of ​​Cabin Awareness came back in 2019 at the first Toyota Connected Hackathon, a 36-hour innovation event that challenged teams of software designers and engineers to develop and test real-world solutions . After winning the hackathon, the idea quickly gained traction within Toyota, allowing Roberts to build an engineering team and further develop the technology.

After evaluating several mmWave vendors, the Toyota team began working with Vayyar Imaging’s in-cab monitoring platform.

Vayyar’s single-chip, high-resolution 4D imaging radar “is the only solution on the market capable of monitoring a vehicle’s entire cabin with a single sensor,” says Toyota, offering “cloud-based imaging exceptionally dense dots”.

Join the list.
Dallas innovates, every day.

Sign up to keep an eye on what’s new and coming to Dallas-Fort Worth, every day.


  • High school students who focused on the mental and emotional aspects of healthcare dominated the hackathon. That’s especially impressive, considering that of the 14 teams competing, four were high school teams, eight were college teams, and two were pro teams. See the top three winning submissions here.

  • High-Tech High Heels North Texas partner will receive a $500,000 grant from Toyota's American Foundation.

    The North Texas nonprofit will receive half a million dollars from the Toyota Foundation USA to help increase the number of girls entering a college STEM program.

  • The program is based on success at West Dallas Stem School, supported by approximately $5 million in donations from Toyota and a collaboration between Dallas ISD and SMU. Now, Toyota is applying the STEM-focused educational model across the country, to help prepare K-12 students for the job market, while offering a variety of other community engagement activities.

  • The “S-AM” hybrid electric minivans – short for “Sienna Autonomo-MaaS” – will be tested in a six-month pilot program in Dallas, Pittsburgh, the San Francisco Bay Area and other places. Aurora intends to roll out the vehicles to ride-sharing networks in late 2024.

  • Aurora wrapped up a six-month pilot trial of its self-driving Toyota S-AM minivans in Dallas-Fort Worth by giving Toyota North America CEO Ted Ogawa and other executives a ride to the airport yesterday DFW. Aurora will continue to test and develop its transit and autonomous trucking technology from Dallas, including a new trucking route from Dallas to El Paso. It plans to launch its ride-sharing product, Aurora Connect, in late 2024.


About Author

Comments are closed.