Publication

Christos Sakaridis, Dengxin Dai, and Luc Van Gool
Semantic Foggy Scene Understanding with Synthetic Data
Submitted for publication
[PDF]   [arXiv]   [BibTeX]  

Foggy Datasets

We present two distinct datasets for semantic understanding of foggy scenes: Foggy Cityscapes and Foggy Driving.

Foggy Cityscapes derives from the Cityscapes dataset and constitutes a collection of synthetic foggy images generated with our proposed fog simulation that automatically inherit the semantic annotations of their real, clear counterparts. Due to licensing issues, the main dataset modality of foggy images is only available for download at the Cityscapes website. This is also the case for semantic annotations as well as other modalities which are shared by Foggy Cityscapes and Cityscapes. On the contrary, the auxiliary Foggy Cityscapes modalities of denoised depth maps and transmittance maps are available at this website in the following packages:


Transmittance maps (8-bit) for foggy scenes in train, val, and test sets
15000 images (5000 images x 3 fog densities)
MD5 checksum

Transmittance maps (8-bit) for foggy scenes in trainextra set
19997 images
MD5 checksum

Depth maps (denoised and complete) for train, val, and test sets
5000 files (MATLAB MAT-files)
Download credentials    MD5 checksum

Depth maps (denoised and complete) for trainextra set
19997 files (MATLAB MAT-files)
Download credentials    MD5 checksum

Foggy Driving is a collection of 101 real-world foggy road scenes with annotations for semantic segmentation and object detection, used as a benchmark for the domain of foggy weather. We provide dense, pixel-level semantic annotations of these images for the 19 evaluation classes of Cityscapes. Bounding box annotations for objects belonging to 8 of the above classes that correspond to humans or vehicles are also available.

Foggy Cityscapes

We develop a fog simulation pipeline for real outdoor scenes and apply it to the complete set of 25000 images in the Cityscapes dataset to obtain Foggy Cityscapes. We also define a refined set of 550 training+validation Cityscapes images out of the original 3475 ones, which yield high-quality synthetic fog. The resulting collection of 550 foggy images is termed Foggy Cityscapes-refined.

We provide three different versions of Foggy Cityscapes for the 5000 training+validation+testing Cityscapes images, each characterized by a constant attenuation coefficient which determines the fog density and the visibility range. The values of the attenuation coefficient are 0.005, 0.01 and 0.02m-1 and correspond to visibility ranges of 600, 300 and 150m respectively. For the 20000 extra training Cityscapes images, we provide a single version with attenuation coefficient of 0.01m-1. Examples of Foggy Cityscapes scenes for varying fog density are shown below.

clear weather 600m visibility 300m visibility 150m visibility

Foggy Driving

Foggy Driving consists of 101 color images depicting real-world foggy driving scenes. 51 of these images were captured with a cell phone camera in foggy conditions at various areas of Zurich, and the rest 50 images were collected from the web. The maximum image resolution in the dataset is 960x1280 pixels.

Foggy Driving features pixel-level semantic annotations for the set of 19 classes that are used for evaluation in Cityscapes. Individual instances of the 8 classes from the above set which correspond to humans or vehicles are labeled separately, which affords bounding box annotations for these classes. In total, Foggy Driving contains more than 500 annotated vehicles and almost 300 annotated humans. Given its moderate scale, this dataset is meant for evaluation purposes and we recommend against using its annotations to train semantic segmentation or object detection models. Example images along with their semantic annotations as well as overall annotation statistics are presented below.

Results

We show that our partially synthetic Foggy Cityscapes dataset can be used per se for successfully adapting modern convolutional neural network models to the condition of fog. Our experiments on semantic segmentation and object detection evidence that fine-tuning “clear-weather” Cityscapes-based models on Foggy Cityscapes improves their performance significantly on the real foggy scenes of Foggy Driving.

Citation

Please cite our publication if you use our datasets or code in your work.

Moreover, in case you use the Foggy Cityscapes dataset, please cite additionally the Cityscapes publication.