Instance-Level Segmentation for Autonomous Driving
with Deep Densely Connected MRFs

People

Ziyu Zhang, Sanja Fidler, Raquel Urtasun

Abstract

Our aim is to provide a pixel-wise instance-level labeling of a monocular image in the context of autonomous driving. We build on recent work [Zhang et al., ICCV15] that trained a convolutional neural net to predict instance labeling in local image patches, extracted exhaustively in a stride from an image. A simple Markov random field model using several heuristics was then proposed in [Zhang et al., ICCV15] to derive a globally consistent instance labeling of the image. In this paper, we formulate the global labeling problem with a novel densely connected Markov random field and show how to encode various intuitive potentials in a way that is amenable to efficient mean field inference [Krähenbühl et al., NIPS11]. Our potentials encode the compatibility between the global labeling and the patch-level predictions, contrast-sensitive smoothness as well as the fact that separate regions form different instances. Our experiments on the challenging KITTI benchmark [Geiger et al., CVPR12] demonstrate that our method achieves a significant performance boost over the baseline [Zhang et al., ICCV15].

Paper

Ziyu Zhang, Sanja Fidler, Raquel Urtasun.
Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs
Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, U.S.A., 2016
[arXiv]

Code and Data

Please visit the Bitbucket repository.

Results

Results vs. Number of Iterations of Mean Field

Acknowledgements

This work was partially supported by ONR-N00014-14-1-0232, Samsung and NSERC.

Contact

For questions regarding the data or code, please contact Ziyu Zhang.