Highway LiDAR + Camera Synchronization Pipeline
What it does
A production-direction pipeline for fixed roadside LiDAR + camera deployed at New York City highway sites. The system handles the full loop:
- Multi-sensor capture of LiDAR sweeps and RGB camera streams.
- Temporal synchronization between LiDAR and camera clocks (timestamp alignment, drift correction).
- Spatial calibration of LiDAR ↔ camera extrinsics for 2D–3D fusion.
- Background subtraction + segmentation on synchronized frames.
- CNN-based 3D detection classifying vehicles, motorcycles, bicycles, and pedestrians (vulnerable road users).
- Multi-frame vehicle reconstruction that stitches successive sweeps into per-vehicle point cloud models.
Why synchronization is the hard part
If LiDAR and camera disagree by even tens of milliseconds, point projection drifts, multi-frame reconstruction smears, and downstream detector quality collapses. A lot of the engineering is the unglamorous time-sync / calibration layer that everything else relies on.
Configuration portability
I built the pipeline so models trained on one site’s sensor configuration can be redeployed on another — different beam counts, different mounting geometries, different frame rates — without rewriting the inference path.
Outputs
- Two MobiSPC 2025 papers (sensing perspectives survey + LiDAR beam-count study for VRU detection).
- Field-deployed pipeline supporting NYC DOT collaboration through the AI & Mobility Research Lab at CCNY.