Health Informatics

Researchers from the Vision Research Laboratory at the University of California, Santa Barbara, in collaboration with Dr. Jeffrey C. Fried of Santa Barbara Cottage Hospital, combine computational methods with clinical expertise to evaluate and improve patient care. The goal is to build technologies that assess existing therapies and generate the clinical evidence needed to devise new ones.

Working inside a real intensive care unit imposes hard constraints: systems must protect the privacy of patients and staff, stay non-intrusive and non-disruptive, and fit within existing hospital infrastructure and standards of care. The team's initial efforts focus on analyzing patient poses and motion — signals tied to sleep hygiene, the prevention of decubitus ulcers (bed sores), and clinical workflow.

The aim is not just to observe patients, but to turn everyday ICU sensing into clinical evidence that can validate today's therapies and shape tomorrow's.

Our approach

The work is built on a multimodal, multiview sensing network. By fusing several complementary sensors and viewpoints, the algorithms remain reliable under the messy realities of a hospital room — partial occlusions, shifting illumination, and privacy limits on what can be recorded.

  • Multimodal, multiview sensing — RGB, depth, and thermal cameras with room environmental sensors
  • Privacy-preserving and non-intrusive by design, protecting both patients and staff
  • Built to fit existing ICU infrastructure and standards of care
  • Robust to real-world conditions such as occlusions and lighting changes

Clinical applications

Patient pose and motion carry clinically meaningful information. The team's methods target three areas where automated analysis can support care:

  • Sleep hygiene — assessing the quality and continuity of patient sleep
  • Decubitus ulcers — tracking poses and repositioning to reduce pressure injuries (bed sores)
  • Clinical workflows — monitoring visits, sanitation, and the delivery of care

Systems

Eye-CU — sleep pose classification from multiview data

Eye-CU classifies patient sleep poses using purely visual sensors — RGB and depth cameras placed at different viewpoints — removing the need for a costly, intrusive pressure mat. A coupled-constrained least-squares (cc-LS) modality-trust algorithm learns how much to rely on each sensor and view, keeping classification accurate even under variable illumination and partial sensor occlusion.

Deep Eye-CU (DECU) — summarizing patient motion in the ICU

Deep Eye-CU adds temporal motion modeling to the multimodal multiview network to summarize how patients move over time. It represents motion as a state machine — drawing on hidden Markov models and pose time-series — and uses state duration to separate true resting poses from brief transitional ("pseudo") poses. DECU has been deployed in a working medical ICU at Santa Barbara Cottage Hospital, with consented study volunteers.

MEYE / Mock-ICU — the multimodal ICU sensing network

The MEYE network fuses RGB, depth, and thermal cameras with a flexible pressure-mat sensor array and room environmental sensors (temperature, humidity, and sound). Combining these modalities lets the algorithms cope with challenging scene conditions such as partial occlusions and lighting changes, with the room sensors used to trigger and tune each modality's contribution. A dedicated mock-ICU setup lets the team develop and validate methods before deployment in the live unit.

Collaboration

Health Informatics is a partnership between the UCSB Vision Research Lab, led by Professor B.S. Manjunath, and the intensivists and medical practitioners of the Medical Intensive Care Unit (MICU) at Santa Barbara Cottage Hospital, working with Dr. Jeffrey C. Fried. The collaboration pairs computer-vision and machine-learning expertise with front-line clinical practice, under protocols designed to protect and maintain the privacy of patients and staff throughout.

Related publications