Start the RealSense IMU pipeline before the video pipeline - #3670
Start the RealSense IMU pipeline before the video pipeline#3670jeff-hykin wants to merge 1 commit into
Conversation
On a D455 the motion module fails to open as a second pipeline with "No device connected" once the video pipeline holds the device (even with enable_device(serial)). The reverse order works, so start the IMU pipeline first and query stream rates from the device context instead of the not-yet-started video profile.
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## main #3670 +/- ##
==========================================
+ Coverage 77.30% 77.56% +0.26%
==========================================
Files 1287 1281 -6
Lines 123173 122262 -911
Branches 10821 10737 -84
==========================================
- Hits 95216 94832 -384
+ Misses 24967 24339 -628
- Partials 2990 3091 +101
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 26 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Greptile SummaryThis change starts the RealSense IMU stream before video initialization for devices that require motion streams to be opened first. Focused executions reproduced two reliability failures in Confidence Score: 3/5Not safe to merge until RealSense startup consistently selects one physical device and releases the IMU resource after a later startup failure. Focused executable harnesses directly exercised both affected control paths: heterogeneous-device capability discovery selected unsupported rates for the pipeline-selected device, and a video startup exception retained the IMU resource and blocked retry. Files Needing Attention: dimos/hardware/sensors/camera/realsense/camera.py
 T-Rex LogsWhat T-Rex did
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| for device in rs.context().query_devices() | ||
| if not self.config.serial_number | ||
| or device.get_info(rs.camera_info.serial_number) == self.config.serial_number |
There was a problem hiding this comment.
When serial_number is unset, _stream_rates unions IMU capabilities from every connected device while _start_imu leaves its pipeline unbound. On heterogeneous cameras, the pipeline can select a device that does not support the aggregated gyro or accelerometer rate, causing startup to fail; IMU and video can also resolve to different cameras. Select a single device before discovering rates and bind both pipelines to its serial, or reject ambiguous multi-device startup.
Artifacts
Focused RealSense heterogeneous-device reproduction harness
- This executable harness extracts the affected method bodies and simulates two devices with distinct IMU rates, providing a reproducible demonstration of the mismatch.
Aggregate capability discovery with unbound device selection
- The executed baseline shows the methods aggregate 400 Hz gyro and 200 Hz accelerometer support across devices even though selected device A only offers 200 Hz and 100 Hz.
Unbound IMU startup rejects cross-device rates
- The executed startup path passes the aggregate rates into the unbound pipeline and receives an unsupported-stream failure for device A, confirming the bug.
RealSense SDK availability probe
- The environment probe shows `pyrealsense2` is unavailable, so validation used the focused executable SDK simulation rather than attached hardware.
| if self.config.enable_imu: | ||
| self._start_imu() |
There was a problem hiding this comment.
start() opens the IMU pipeline before video startup but does not roll it back if video startup or subsequent initialization raises. The IMU device remains held, so a retry or another camera consumer fails until it is manually stopped. Stop and clear the IMU pipeline, along with any partially started video pipeline, before re-raising the startup error.
Artifacts
Focused RealSense IMU startup failure harness
- A mocked RealSense harness executes the actual AST-extracted `RealSenseCamera.start` method with video startup failure and retry scenarios, demonstrating the unhandled resource rollback.
IMU remains open after video startup failure
- The unmodified start-path run shows a simulated video failure, no IMU stop call, and a retry rejected because the IMU device resource is still open.
Manual IMU cleanup permits retry
- The cleanup control run manually stops the leaked IMU after the same video failure and then successfully retries startup, confirming the required rollback behavior.
github-actions
Bot
added
the
ready-to-merge
1 participant
D455: the motion module fails to open as a second
rs.pipelineonce the video pipeline holds the device, even withenable_device(serial). Reverse order works. SoRealSenseCamera.startfailed wheneverenable_imu=True.Repro (swap the two pipelines and both stream):
start()now starts the IMU pipeline first_stream_ratesqueries the device fromrs.context()(filtered byserial_numberwhen set) since the video profile doesn't exist yetVerified on Alfred's orin-nx-7837: