Capturing the Moment
How smart sensors allow you to focus on the things that count
Capturing the moment and telling a story with a single image – that’s the passion of photographer Johannes Arnold. When searching for exciting subjects on the streets of Berlin, he has to fully focus on the surroundings and keep his hands free. This is made possible by his trusted companions: smart hearables. The built-in sensors from Bosch Sensortec allow users to make phone calls with ease and to operate smart devices by means of gestures or voice input.
Precise detection of head movements
The sky over Berlin is overcast this morning. Johannes Arnold got up early, as he wants to capture the city in the various moods of the light as they change over the course of the day. The cool weather in the German capital takes him by surprise. Will the weather hold today? He looks up at the sky with concern. The gesture is recognized by the BHI360 in his hearables. The smart sensor system from Bosch Sensortec is a game changer. It is not only the smallest system in the class of 16-bit 6-axis IMUs, but also the most powerful. Drawing on the integrated sensor fusion software (BSX), it accurately detects head movements and determines the orientation vector. As Johannes Arnold holds his gaze for a few seconds, the head tracking signal is processed by his weather app. “The sky in Berlin is overcast today, but it will remain dry,” he is informed via the headphones. So he’s good to go. The reflections of the dawning day in the glass facades of a high-rise building become his first motif.
BHI360
The BHI360 redefines performance, power consumption, and space requirements of inertial measurement units (IMU). The sensor system from Bosch Sensortec combines a gyroscope with an accelerometer and two integrated microcontrollers (MCU). The first MCU can execute simpler sensor processing algorithms in a power-saving manner, such as gesture recognition, activity, or step counting. The second MCU is equipped to run more complex algorithms thanks to a high-performance 32-bit microcontroller with integrated 256-kB SRAM memory.
Integrated sensor fusion software enables accurate head tracking that results in realistic and personalized sound experiences in 3D audio and facilitates smooth gesture recognition. Owing to its outstanding energy efficiency, compact design, and integrated algorithms, the BHI360 is particularly suitable for portable battery-powered devices, such as wearables and hearables.
The BHI380 is another product variant. It is based on the same hardware platform but features a comprehensive software package including self-learning artificial intelligence. Not only does it track a variety of fitness activities, but it can train specific exercises to accurately capture personalized workouts – and not just in the gym: Special swim tracking software measures the athlete’s swimming style and performance in the water. For more accurate positioning, such as when the user is performing during outdoor activities, the BHI380’s built-in Pedestrian Dead Reckoning (PDR) algorithm helps guide the user to their destination even if the GPS signal fails. Because of its additional integrated algorithms, the BHI380 is particularly suitable for smart portable devices such as smartwatches and fitness trackers.
Perfect sound when listening and speaking
One of Johannes Arnold’s destinations is just outside the city center. On the streetcar, he relaxes to the sound of music on his headphones. The Adaptive Active Noise Cancelling (ANC) function effectively filters noise during the ride. When he arrives at his stop, the hearables identify the new situation. Moving on foot activates the appropriate ANC profile so that Johannes Arnold can safely pass the heavy traffic of Federal Highway 2.
The prominent road cuts through the western part of the city in a straight line. “From here you can see the TV tower and the Brandenburg Gate – the weather permitting, that is,” he explains. A great motif, and one he had expected. However, “Street photography thrives on the moment, on improvisation.” At the edge of a pedestrian island, he crouches down, looking for his subject in the flow of traffic. He then suddenly stops and nimbly taps his ear twice to take a call that interrupts his work. The BHI360 also supports a new level of human-machine interaction based on optimized algorithms: Multi-tap detection makes it easy to execute commands directed at the hearables.
50% smaller and significantly more powerful
than its predecessor. The BHI360, the smallest IMU on the market, allows a lot of freedom in the design of next generation hearables and wearables.
Johannes Arnold is on the phone in the midst of Berlin traffic. However, the person he is talking to doesn’t hear the engines starting up behind him as the light turns to green. A highly sensitive accelerometer, in combination with a MEMS microphone, translates the skull vibrations generated during speech into clear voice sound, enabling voice enhancement without the interference of ambient noise. Johannes Arnold greatly appreciates convenience functions such as this in his everyday life. “When I’m on the go, I mainly control my devices via voice control. No matter how noisy it is around me, I can fully focus on my surroundings.”
Trend-setting sensor functions
Audio navigation guides Johannes Arnold to the next location. His destination is a spectacular building from the 1970s situated on Messedamm. On the way, he crosses a long underpass. This poses a challenge to the route guidance on his smartphone, but yields a pleasant new discovery for the photographer: “This is a fantastic location.” Bright orange columns support the low ceiling, and round lampshades illuminate the graffiti created by Berlin’s street art scene. Johannes Arnold takes his time, making several shots as he explores the underpass. The determination of his location remains precise, so he can resume the originally set route at any time. This is made possible by the PDR navigation software of the BHI380, which performs sensor-based positioning. The BMM350 ensures that the navigation remains keyed to the geomagnetic north.
BMM350
The BMM350 is a very small, powerful, and low-power 16-bit/3-axis magnetometer based on the unique TMR technology, a Bosch Sensortec innovation. Magnetometers work like a compass and provide accurate orientation with respect to the Earth’s magnetic field. The BMM350 now also offers a variety of new applications, including head orientation detection for 3D audio and pixel latency reduction in AR/VR to improve the user experience and to prevent motion sickness. In indoor navigation scenarios, it provides directional information and, in combination with an IMU such as the BHI360, improves position and speed detection for e-bikes, other vehicles, or industrial applications.
A unique function ensures particular reliability: The “magnetic reset” can tolerate magnetic fields of up to 100 times the measuring range without loss of performance. This allows the sensor to recover from the effects of short-term strong magnetic fields, making it particularly robust.
Sensors that deliver top energy efficiency
Inspired, Johannes Arnold leaves the underpass at the right exit in the direction of his destination. The International Congress Center Berlin is a retro-futuristic behemoth, wrapped in a silver-gray aluminum facade. “I discovered this place while driving by and just had to return to take pictures here,” he confesses. He stands in front of two huge exhaust shafts, with the highway roaring behind him. “Now it’s the right time for some music,” he says laughingly as he presses “play” and starts taking pictures. Thanks to the impressive energy efficiency of the small Bosch Sensortec sensors, he doesn’t have to worry about the batteries in his hearables.
20 times less energy
is consumed by the BMM350 compared to the previous generation, despite a considerably increased scope of functions.
Entertainment on a new level
After sunset, the lights of the city are reflected in the Spree River. Back at his accommodation, Johannes Arnold relaxes from a long day out. On his laptop, he watches the new episode of his favorite show while enjoying the rich 3D sound of his hearables. For this purpose, the BMM350 stores the viewing direction toward the screen via magnetic field recognition and in combination with the BHI360’s sensor fusion software. Together with the IMU, the hearables thus generate a personalized sound experience adapted to the user’s head movements.
This deep link with the laptop is what also causes the series to pause via head tracking when Johannes Arnold moves away from the screen to get a glass of water. “And when I sit down again, playback continues. That way, I never miss anything, even if I have to rush to the door to receive a delivery,” he explains. The ability to focus on what matters at any given moment – this is the promise of the versatile sensors from Bosch Sensortec.
