Contactless Respiratory Monitoring on Heterogeneous Mobile Robots: A Multimodal Edge-Computing Framework

1. Introduction: The High Stakes of Remote Respiratory Monitoring

Respiratory Rate (RR) is a foundational vital sign, serving as a leading indicator of physiological distress, clinical deterioration, and life-threatening conditions in emergency medicine. In the chaotic environments of disaster recovery, such as structural collapses or biohazard containment zones, traditional contact-based sensors—chest straps, nasal probes, or wearables—are fundamentally untenable. Physical contact introduces secondary risks to responders and complicates the speed of triage in time-critical windows.

Mobile robots equipped with contactless monitoring offer a solution to this access problem, yet they introduce a new set of risks regarding data integrity. This document analyzes a multimodal edge-computing framework designed to operate as a Risk Mitigation Layer. By integrating diverse sensing modalities and rigorous signal validation, the framework prevents the “hallucination” of vital signs—a core AI safety concern—ensuring that triage decisions are based on verified physiological signals rather than environmental noise.

2. The Modality-Adaptive Framework: A Multi-Stage Pipeline

The system architecture follows a platform-agnostic, multi-stage pipeline designed for architectural resilience across heterogeneous robotic hardware.

1. Brightness-Adaptive Sensor Selection: The framework initiates by calculating the mean brightness ($B$) of the detection frame. Using a deterministic threshold of $B \le 60$, the system autonomously switches from RGB to a secondary modality (NIR, Thermal, or Low-Light) to maintain the Operational Envelope in degraded visibility.
2. 2D Pose-Based ROI Extraction: The system utilizes YOLOv11x-pose for chest tracking. The “n” (nano) variant was explicitly rejected for this safety-critical application due to lower keypoint confidence and systematic detection failures at distances $>6\text{m}$, particularly in non-upright postures. By deriving the ROI from shoulder and hip keypoints, the system remains robust to sitting or leaning poses.
3. Signal Extraction and Filtering: For RGB, the system isolates the green channel to capture thoracic expansion. For Thermal, NIR, and Low-Light modalities, grayscale outputs are used, where the mean pixel intensity across 64 grid points is extracted. Signals are processed through a fifth-order Butterworth filter (8–35 BPM) and ranked by energy.
4. SQI-Filtered Sliding Window Analysis: The framework applies a 15-second sliding window with a 5-second step. A Signal Quality Index (SQI) serves as a Reliability Gating mechanism, rejecting data windows that fail to meet strict periodicity and spectral concentration requirements.
5. Harmonic Disambiguation: To ensure temporal consistency, the system evaluates subharmonic energy at $f/2$ and $f/3$. A deterministic correction logic is applied: if the subharmonic frequency carries more than 30% of the dominant peak power, the estimate is corrected to prevent frequency-doubling errors.

3. Sensor Synergy: Choosing the Right Tool for the Environment

Effective triage requires understanding the physical boundaries of each sensing technology.

Modality	Effective Range (m)	Primary Environmental Use Case	Key Limitation
RGB	Up to 8m	Illuminated indoor/outdoor zones	Requires ambient light ($B > 60$)
Thermal	Up to 2m	Complete darkness / Smoke	Resolution-limited (FLIR C5)
NIR	Up to 6m	Dim/Dark indoor environments	Active illumination (850nm) falloff
Low-Light	Up to 8m	Passive extreme low-light (0.0001 lux)	Lying-pose geometry failures > 6m

Hardware Implementation: The framework was validated using the FLIR C5 for thermal imaging and a custom array of six 850nm LEDs for active Near-Infrared illumination.

4. Failure Modes and Robustness Implications

A safety-first architectural assessment requires identifying the deterministic failure boundaries where the system can no longer guarantee accuracy.

The Geometry Constraint: Mounting Height vs. Subject Posture

The system exhibits a deterministic failure boundary in “lying-pose” scenarios beyond 6m. Because the cameras are mounted at approximately 0.6m (payload height), prone subjects present a significantly foreshortened torso. This reduction in pixel separation between shoulder and hip keypoints prevents reliable ROI extraction. For future deployments, practitioners must consider tilt-actuated gimbals or increased sensor elevation to mitigate this geometric blind spot.

Thermal Resolution Bottlenecks

The use of the FLIR C5 introduces a rigid range constraint. Beyond 2m, the sensor’s native resolution provides insufficient pixel density within the chest ROI for YOLOv11x-pose to maintain keypoint lock. This represents a hardware-induced failure mode where the pipeline terminates at the detection stage despite a clear heat signature.

Active Illumination Falloff

The NIR operational envelope is strictly governed by the inverse-square law. At 8m, the fixed-power 850nm LED array fails to provide a sufficient signal-to-noise ratio (SNR). Signals at this range are consistently caught by the SQI gate and rejected, preventing the reporting of low-confidence data.

5. Edge Computing Performance Across Heterogeneous Platforms

Computational bottlenecks directly impact the latency of triage decisions. The framework was tested across three distinct compute architectures:

• Platform A (Spot / Jetson Orin AGX): The gold standard for efficiency, utilizing only 9.59% CPU and 11.53% GPU. It provides the most stable mission profile with a 22W power draw.
• Platform B (Vision 60 / Jetson Xavier): Represents a resource-constrained environment, exhibiting 69.8% CPU and 35% GPU load. The 2-minute 20-second execution time indicates a critical processing bottleneck for real-time triage.
• Platform C (Husky A300 / RTX 4060 Ti): Achieves peak speed (1 min 34 s) but incurs a heavy power penalty (38–65W), highlighting the trade-off between execution speed and operational battery life.

6. The Signal Quality Index (SQI): The Gatekeeper of Accuracy

The SQI is the primary defense against vital sign hallucination. It is calculated as: $SQI = 0.3(1 - SF) + 0.4 \min(1, P/10) + 0.3R_{ac}$

Spectral Flatness (SF) is defined as the Wiener entropy, representing the ratio of the geometric mean to the arithmetic mean of the power spectrum: $SF = \frac{\exp\left(\frac{1}{K} \sum_{k=1}^K \ln S(k)\right)}{\frac{1}{K} \sum_{k=1}^K S(k)}$

The framework employs modality-specific safety gates to account for inherent noise floors (per Table I):

• RGB (Platform A): Strict Flatness threshold ($< 0.75$).
• NIR / Platform B & C: Moderate Flatness threshold ($< 0.85$).
• Low-Light: Lenient Flatness threshold ($< 0.88$) to accommodate passive sensor noise in extreme darkness.

7. Conclusion: Lessons for Real-World Deployment

Multimodal robotic triage is feasible but requires a rigorous adherence to operational boundaries to ensure patient safety.

Critical Takeaways for Practitioners:

1. Modality Redundancy is Mandatory: Single-sensor systems are prone to catastrophic failure in the dynamic lighting of disaster zones.
2. Keypoint-Guided ROIs are Superior: Traditional bounding boxes fail in non-upright postures; pose estimation is required for reliable chest tracking.
3. SQI-Gating is Non-Negotiable: Systems must be designed with the agency to reject their own data to avoid providing false triage information.
4. Mounting Height vs. Subject Posture Alignment: Ground-level robots possess a geometric blind spot for prone victims at range, necessitating adaptive sensor positioning.

Future Directions: Research is shifting toward the integration of heart-rate data and the development of motion-compensation algorithms. Crucially, implementing adaptive robot positioning—where the robot autonomously adjusts its stand-off distance and angle to optimize ROI geometry—will be essential to overcoming the current limits of remote monitoring.

Read the full paper on arXiv · PDF