AR-CNN: an attention ranking network for learning urban perception - Li, Zhetao; Chen, Ziwen; Zheng, Wei Shi; Oh, Sangyoon; Nguyen, Kien (Science China Information Sciences, 2022)
An increasing number of deep learning methods is being applied to quantify the perception of urban environments, study the relationship between urban appearance and resident safety, and improve urban appearance. Most advanced methods extract image feature representations from street-level images through conventional visual computation algorithms or deep convolutional neural networks and then directly predict the results using features. Unfortunately, these methods take color and texture information together during processing. Color and texture are prime image features, and they affect human perception and judgment differently. We argue that color and texture should be operated differently; therefore, we formulate an end-to-end learning methodology to process input images according to color and texture information before inputting it into the neural network. The processed images and the original image constitute three input streams for the triad attention ranking convolutional neural network (AR-CNN) model proposed in this study. In accordance with the aspects of color and texture, an improved attention mechanism in the convolution layer is proposed. Our objective is to obtain the scores of humans on urban appearance in accordance with the prediction results computed from pairwise comparisons generated by the AR-CNN model.
Keywords: Attention network; Attribute learning; Color and textures; Color information; Convolutional neural network; Learning methods; Neural network model; Ranking network; Texture information; Urban perception; attention network; attribute learning; colour and texture; ranking network; urban perception