PASCAL VOC Segmentation Challenge 2009

CVC Team

Xavier Boix, Josep M. Gonfaus, Fahad Shahbaz Khan, Joost Van de Weijer, Andrew D. Bagdanov, Marco Pedersoli, Jordi Gonzàlez, Joan Serrat.

Slides can be downloaded from here.

Experimental Results

Our method obtained the best score in 6 of the 20 classes, thereby finishing second behind the Bonn submission. More results can be found at the PASCAL VOC2009 workshop page.

The results show that harmony potentials are able to deal with multiclass images, partial occlusion, and to correctly classify the background.


We use a Conditional Random Field (CRF) approach. The main novelty is the introduction of a new potential, called harmony potential, which allows to encode any combinations of labels at the global node. This improvement is especially relevant for the larger scale regions in the image, and allows us to exploit to results of our image classification method. The method is explained in detail in our CVPR2010 paper.

Our model is a two-level CRF that uses labels, features and classifiers appropriate to each level. The lowest level of nodes represents superpixels labeled with single labels, while a single global node on top of them permits any combination of primitive local node labels. A new consistency potential, which we term the harmony potential, is also introduced which enforces consistency of local label assignment with the label of the global node. We propose an effective sampling strategy for global node labels that renders tractable the underlying optimization problem.


Josep M. Gonfaus, Xavier Boix, Joost Van de Weijer, Andrew D. Bagdanov, Joan Serrat, and Jordi Gonzàlez, " Harmony Potentials for Joint Classification and Segmentation ", in Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, 2010.

Lubor Ladicky, Chris Russell, Pushmeet Kohli, and Philip H.S. Torr. Associative hierarchical crfs for object class image segmentation ", in International Conference on Computer Vision (ICCV), Kyoto, Japan, 2009.

Nils Plath, Marc Toussaint, and Shinichi Nakajima. " Multi-class image segmentation using conditional random fields and global classification ", in International Conference on Machine Learning (ICML), Montreal, Canada, 2009.