Abstract

Classical occupancy grid maps encode space as a binary field: free or occupied. This representation suffices for obstacle avoidance but provides no context about what objects occupy which cells, making it difficult for a mobile robot to reason about navigability, object affordances, or task-relevant regions. Semantic occupancy grids augment each cell with a categorical label and a confidence distribution over that label, bridging geometric mapping and scene understanding. This article surveys the architecture, training approaches, and deployment considerations for semantic occupancy grids in practical indoor mobile robot systems as they stand in mid-2025.

From Geometric to Semantic Maps

The earliest semantic mapping systems simply ran a 2D object detector on RGB images and projected detections into an existing metric map using known camera geometry. This approach is brittle: overlapping objects, occlusion, and detector label noise produce inconsistent maps that degrade navigation. More recent systems treat semantic and geometric mapping as a joint inference problem.

SemExp, NaviSem, and related architectures from Habitat Challenge 2023-2024 demonstrated that a recurrent policy trained on semantically annotated scans (Matterport3D, Gibson) can maintain an implicit semantic map in its hidden state, predicting both free space and object categories simultaneously. The shift toward explicit voxel or BEV (Bird’s Eye View) representations then brought interpretability: the robot’s beliefs become inspectable grids rather than opaque network activations, important for deployment in safety-critical settings such as hospital corridors or warehouses shared with human workers.

Architecture and Training

Current state-of-the-art systems use a BEV encoder-decoder that takes multi-view RGB-D observations and outputs a top-down grid with per-cell semantic logits. Lift-Splat-Shoot style view transformers, originally developed for autonomous driving perception, have been adapted effectively to indoor settings by groups at Stanford and ETH Zurich. The key adaptation is handling the denser occlusion patterns in indoor environments, where shelves and furniture block large fractions of the robot’s field of view from any single pose.

Training requires semantically annotated 3D scans. The ScanNet and ARKitScenes datasets provide this at scale, with 200-class vocabularies that cover furniture, appliances, and structural elements common in homes and offices. Knowledge distillation from large vision-language models such as CLIP and OpenSeg improves zero-shot category generalization, allowing the semantic grid to label objects not present in the training scans without explicit annotation.

Deployment Considerations

Inference latency is the dominant engineering constraint. A full BEV semantic prediction at 0.05-meter resolution over a 20x20 meter room requires processing several thousand voxels per forward pass. On an NVIDIA Jetson AGX Orin running a quantized INT8 model, current architectures achieve 8-12 Hz, adequate for navigation speeds below 1 m/s but marginal for faster platforms. TensorRT-optimized models and selective update strategies, refreshing only cells within the sensor frustum, push this to acceptable rates for most logistics platforms.

Map persistence across sessions remains an open problem. Lighting changes, furniture rearrangement, and seasonal decor shifts cause previously mapped semantic labels to become stale or incorrect. Online map correction triggered by prediction confidence drops, combined with efficient incremental re-labeling of changed regions, is an active research area with practical prototypes demonstrated by the mobile robotics group at KIT Karlsruhe.

Conclusion

Semantic occupancy grids represent a practical and increasingly mature middle ground between purely geometric maps and fully implicit neural scene representations. Their explicit, inspectable structure makes them amenable to integration with task planners and safety monitors. As inference hardware improves and training datasets broaden, the gap between what a mobile robot can perceive geometrically and what it can understand semantically will continue to close.