Entropy-weighted Voting Method for Diffusion-based Semantic Segmentation - Jeong, Seonggyun; Heo, Yong Seok (International Conference on ICT Convergence, 2023)
Recently, semantic segmentation methods leveraging image generation models have garnered significant attention. In particular, approach based on Diffusion models (DDPM) that utilize mid-level activations from the diffusion network with a majority voting of distributions from several light multi-layer perceptron (MLP) have shown better performance compared to GAN-based approaches. However, utilizing a simple majority voting system is suboptimal. In this paper, we propose a novel voting method for DDPM-based semantic segmentation. Our method introduces a weighted sum of distributions, where the weights are determined by the entropy of the class prediction results obtained from each MLP model. We conduct experiments on various datasets, including LSUN-Bedroom, FFHQ-256, LSUN-Cat, and LSUN-Horse. The results demonstrate that our proposed method achieves better mean Intersection over Union (mIoU) scores compared to previous work.
Keywords: Diffusion model; Diffusion networks; Image generations; Multilayers perceptrons; Performance; Segmentation methods; Semantic segmentation; Simple majority; Voting; Weighted voting methods; diffusion; entropy; semantic segmentation; voting