Driver monitoring systems are to become compulsory in all new passenger vehicles from 2024. IMVE's Ben Townsend explores the vision tech enabling them
By next year, you may have a new passenger in your car – a driver monitoring system (DMS). In a push to reduce the number of fatalities and injuries on the road, the General Safety Regulation (GSR) has said that all new passenger cars and light commercial vehicles (e.g. pick-up trucks and vans) sold in the European Union (EU) must have a DMS installed by 2024.
The EU is the first to implement such legislation, but other parts of the world aren’t too far behind. Despite not making it through the Senate, the US House of Representatives passed the Moving Forward Act in 2020, which aimed to install a DMS in all new vehicles with automated features. In 2018, the province of Jiangsu in China made DMS mandatory in long-haul trucks and vehicles transporting hazardous materials. Clearly, DMS is making ripples in governments – a sign of the times to come.
The clue’s in the name with driver monitoring systems – they’re designed to monitor a driver’s behaviour and body language, alerting if they show signs of drowsiness or distraction.
Using a series of cameras and sensors, as well as facial recognition, head positioning, and eye-tracking technology, these systems aim to reduce the number of accidents caused by driver error. In turn, alerts are also intended to encourage safer driving habits, as well as extend vehicle lifespan by reducing accidents.
That’s why the photonics industry is jumping on the chance to develop DMS before the market goes full throttle.
VCSEL-based DMS
Optical solutions provider Ams Osram is one of these companies – it has launched several driver-monitoring technologies over the past couple of years.
Speaking to Imaging and Machine Vision Europe at the recent Laser World of Photonics exhibition in Munich, Director of Global Marketing at Ams Osram, Martin Wittmann, said: “We’re seeing more and more sensing and monitoring happening in the car for autonomous driving, but also for applications like driver monitoring – monitoring alertness – which is pending legislation to enter the market.”
He added: “We’re excited about these applications because they improve road safety and will hopefully lead to fewer fatalities. They do something really good for the world. Whether you’re looking or talking on your phone, eating, or just sleeping, the driver monitoring system will know what’s going on and alert you.”
One of these technologies from Ams Osram is the TARA2000-AUT-SAFE, a vertical-cavity surface-emitting laser (VCSEL) emitting a controlled beam of infrared light at 940nm, designed for driver monitoring, gesture sensing, and interior cabin monitoring. The company says it offers a more reliable eye safety feature than existing automotive VCSEL modules, and mass production is planned for 2024, in step with the EU regulation.
“The nice thing about this technology,” Wittmann said, “is that it can shape and distribute laser light. It can see if the driver is wearing their safety belt, as well as how often people are blinking or if they’re responding. It can even tell if they’re looking outside the car.”
Wittmann added: “There are two different solutions: you can either use our traditional LED technology, which is a more cost-effective solution that offers fewer features – or the pixel technology, which is higher-end. And we’re the only one on the market right now who can really develop both chip technologies and can offer them to customers.”
Mass production of Ams Osram’s TARA2000-AUT-SAFE 940nm VCSEL for DMS systems is planned for 2024, in step with new EU regulations (Image: AMS Osram)
When asked about how the technology could deal with the subjectivity of what a driver ‘distraction’ is, Wittman said: “We use very sophisticated algorithms. We work together with a Swedish company called Smart Eye, which is also part of the demonstrator we were building.
“The algorithm is very important. The algorithm shouldn’t have false detection, otherwise people are just going to turn it off.”
Indeed, one challenge with DMS is that drivers may find it invasive or even frustrating, and thereby be reluctant to use it.
Talking about how a driver would be notified, Wittman said: “It’s an audio-visual alert with visible LEDs. For example, the dashboard could turn red to say, ‘Hey, eyes on the road, please!’ There will also be vibrations on the steering wheel.”
Ams Osram also released a proof-of-concept last year called Icarus, which aims to upgrade DMS designs to include features such as an augmented reality head-up display, secure driver authentication, and advanced drowsiness detection. The Icarus system accurately measures the position of the driver’s eyes and head pose in 3D using a depth map, Ams Osram says, and supports software that can detect micro-sleeps, alongside other signs of drowsiness. The same technology could also one day be used to support online shopping features for passengers riding in the back of future autonomous vehicles. “We built a 3D sensing demo that has more functionalities, such as identifying a person to authorise a payment, for example while shopping on Amazon,” said Wittman.
One of the biggest hurdles for DMS manufacturers is cost. The systems can be expensive to install and maintain, which makes rolling them out commercially a challenge. That’s why companies such as Ams Osram are developing compact, cost-effective models.
“You really want to hide [the DMS],” Wittmann says. “Everything needs to look super slick. You don’t want to necessarily tell the drivers ‘Oh, there is a camera sensor monitoring you all the time’. Therefore, the component size is very important. This latest generation is very compact and very powerful.”
3D vision-based DMS
Another organisation working on driver monitoring technology is the Fraunhofer Institute for Applied Optics and Precision Engineering IOF, which has developed a miniature wide-angle 3D camera in collaboration with car manufacturer BMW.
The camera, which attaches to the rear-view mirror, can be used to capture people and objects, as well as their position, movement, and shape in three dimensions. It also offers safety features such as hand-on-steering-wheel detection, seat belt detection, and seat occupancy monitoring coupled with airbag control. Airbag control involves adapting the way the airbag opens, depending on how someone is sitting, to prevent injuries.
Fraunhofer IOF’s miniaturised wide-angle 3D camera can be mounted on the underside of a vehicle’s rear-view mirror
The camera works by emitting light pulses that reflect off of objects and are used to calculate a point cloud. By comparing these point clouds with known shapes, people as well as objects such as child safety seats, can be identified.
Like Ams Osrams’ Icarus proof-of-concept, Fraunhofer IOF is also focusing on hand gestures. With gesture control, drivers can access their car’s dashboard and infotainment features with a wave of their hand. While this makes life easier for drivers when changing a playlist, for example, it also aims to improve safety – by minimising distractions that may come from using buttons or screens.
Again at the Laser World of Photonics, Christian Vetter, Head of Advanced Imaging Design at Fraunhofer IOF, told Imaging and Machine Vision Europe: “The goal is to completely monitor the inside of the car using depth recognition. You can see basically if the kids are waving in the back, or if the person in the front wants to skip to the next track in the music. It can recognise hand gestures.”
Vetter explained that there are hurdles when it comes to developing a DMS, however: “You want to use a time-of-flight sensor. But the problem is that time-of-flight sensors typically have very low resolution. And if you have low resolution, people will be very distorted in the image, which means you have to move your pixels around. And it’s not really possible to compensate for that with a low-resolution sensor.”
A close-up of Fraunhofer IOF's miniaturised wide-angle 3D camera for driver monitoring (Image: Fraunhofer IOF)
Explaining how Fraunhofer IOF and BMW overcame this problem, Vetter said: “We came up with a technology where we have multiple stacked microlens systems to enable full aberration correction. We are redirecting the field of view with the presence outside. This means we have six images of different areas on the sensor, which can be stacked together to form a full image. Because of this arrangement, each individual channel is less distorted than a fisheye objective would be.”
Like Ams Osram, Fraunhofer IOF is striving for compactness, with its wide-angle 3D camera measuring just 10mm across. “The camera can be extremely tiny and goes on the rear-view mirror, and it can look all around the car,” said Vetter.
He added: “The advantage of having just one camera is you just have to route one cable. You could put 20 cameras in the car, but then you’d have to route 20 cables. You don’t want to put 20 cameras in there, because one will break every two weeks. The fewer breaking points, the better!”
“The project, as far as I know, is finished,” he continued. “The camera already survived the temperature tests, from minus 40 to plus 80 degrees.”
So, it seems, DMS technology is not far off. Come next year, the driving experience may be very different, and – hopefully –
far safer.
