Google Introduces SensorLM to Boost Wearable Health Data Interpretation

SensorLM uses contrastive learning and generative pre-training to generate hierarchical and context-aware text captions that describe user activity from sensor signals.

Google has introduced SensorLM, a foundation model trained to analyze wearable sensor data and generate personalized health insights.

The company said the model was built on 59.7 million hours of multimodal data collected with user consent from 103,643 people across 127 countries using Fitbit and Pixel Watch devices between March 1 and May 1, 2024.

In a statement, Google explained the rationale behind the new model: "While we can easily see what our body is doing (e.g., a heart rate of 150 bpm), the crucial context of why (say, 'a brisk uphill run' vs. 'a stressful public speaking event') is often missing."

SensorLM uses contrastive learning and generative pre-training to generate hierarchical and context-aware text captions that describe user activity from sensor signals.

According to Google, the model can differentiate between cardio and strength training activities and produce more accurate captions than non-specialist language models.

"Given only the high-dimensional sensor signals from a wearable device, SensorLM can produce hierarchical and contextually relevant captions. Experimental results indicate that these generated captions were more coherent and factually correct than those produced by powerful non-specialist LLMs," the company stated.

Google also plans to extend the model’s use to more specific domains. "Looking forward, we plan to scale pre-training data into new domains, including metabolic health and detailed sleep analysis, to address the messy reality of consumer health devices," the company said.

Additionally, the company noted future possibilities for SensorLM, saying, "We envision SensorLM leading to a future generation of digital health coaches, clinical monitoring tools, and personal wellness applications that can offer advice through natural language queries, interactions, and generation."

This announcement aligns with Google’s ongoing push to embed AI in its wearable health tech offerings. Earlier this year, the company received FDA clearance for a loss of pulse detection feature on the Pixel Watch 3. That feature uses infrared and red lights, motion sensors, and an AI algorithm to detect pulse loss and alert emergency services if needed.

Meanwhile, Samsung, in partnership with Google, recently launched its Galaxy Watch8 Series, which features health tools like sleep apnea detection and vascular stress monitoring.

Stay tuned for more such updates on Digital Health News