RoGaussianOCC

Accurate and robust Gaussian occupancy prediction through temporal fusion and perspective supervision.

Overview

RoGaussianOCC is a camera-based 3D semantic occupancy-prediction framework for autonomous driving. It uses object-centric Gaussians as a compact scene representation and introduces temporal information to address inefficient Gaussian placement, distribution distortion under occlusion, and sparse supervision during training.

My contribution

  • Proposed and implemented the complete RoGaussianOCC framework.
  • Led model training, parameter tuning, ablation studies, visualization, and project coordination.
  • Authored the paper as a co-first author and prepared the public demo and codebase.

Technical approach

Historical Gaussian fusion
Aligns historical Gaussians with ego motion and fuses prior and current queries to concentrate representation capacity on informative regions.
Gaussian feature aggregation
Samples continuous multi-frame image features in 4D and fuses temporal Gaussian features to recover occluded objects.
Perspective supervision
Adds an auxiliary perspective head that provides denser supervision to the image backbone and improves convergence.
Gaussian-to-occupancy decoding
Iteratively refines object-centric Gaussians and splats them into a semantic occupancy grid for downstream driving perception.
RoGaussianOCC framework with historical Gaussian fusion, 4D feature aggregation, perspective supervision, and Gaussian-to-occupancy decoding
RoGaussianOCC framework. Historical Gaussian queries are aligned and fused with the current frame, multi-scale image features are aggregated across space and time, and perspective supervision directly strengthens the backbone before iterative Gaussian refinement and occupancy decoding.

Results

32.20%best IoU on nuScenes with 51,200 Gaussians
20.65%best mIoU on nuScenes with 51,200 Gaussians
+9.02%relative IoU improvement over GaussianFormer at 25,600 Gaussians
+26.4%relative mIoU improvement over GaussianFormer at 25,600 Gaussians
RoGaussianOCC qualitative comparison with GaussianFormer on rain, occlusion, distant objects, dark scenes, and overlapping objects
Qualitative results on nuScenes. Compared with the single-frame GaussianFormer baseline, RoGaussianOCC recovers occluded and distant vehicles more completely and maintains stronger semantic consistency in rain, darkness, and overlapping-object scenes.
Ablation comparison of RoGaussianOCC occupancy predictions with and without Gaussian feature aggregation
Occlusion robustness. On a manually curated set of 227 occlusion-challenging samples, Gaussian feature aggregation uses historical observations to recover vehicles and trucks that are missed when temporal aggregation is removed.

Paper, demo & code