**Predicting Aesthetic Score Distribution through Cumulative Jensen-Shannon Divergence**

Xin Jin^{1} , Le Wu^{1} , Xiaodong Li^{1} , Siyu Chen^{1} , Siwei Peng^{3} , Jingying Chi^{3} , Shiming Ge^{4} , ∗ , Chenggen Song^{2} , Geng Zhao^{1}

^{1}Department of Computer Sci. and Tech., Beijing Electronic Science and Technology Institute, Beijing, 100070, China

^{2}Department of Info. Sec., Beijing Electronic Science and Technology Institute, Beijing, 100070, China

^{3}College of Info. Sci. and Tech., Beijing University of Chemical Technology, Beijing 100029, China

4Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100093, China

∗Corresponding author email: geshiming@iie.ac.cn

**Abstract**

Aesthetic quality prediction is a challenging task in the computer vision community because of the complex interplay with semantic contents and photographic technologies. Recent studies on the powerful deep learning based aesthetic quality assessment usually use a binary high-low label or a numerical score to represent the aesthetic quality. However the scalar representation cannot describe well the underlying varieties of the human perception of aesthetics. In this work,we propose to predict the aesthetic score distribution (i.e., a score distribution vector of the ordinal basic human ratings)using Deep Convolutional Neural Network (DCNN). Conventional DCNNs which aim to minimize the difference between the predicted scalar numbers or vectors and the ground truth cannot be directly used for the ordinal basic rating distribution. Thus, a novel CNN based on the Cumulative distribution with Jensen-Shannon divergence (CJS-CNN) is presented to predict the aesthetic score distribution of human ratings, with a new reliability-sensitive learning method based on the kurtosis of the score distribution, which eliminates the requirement of the original full data of human ratings (with- out normalization). Experimental results on large scale aesthetic dataset demonstrate the effectiveness of our introduced CJS-CNN in this task.

**Gallery**

**Images with similar mean scores (i.e., around5). The rating distributions are approximated by the score histograms (1-10). The hist., var., skew. and kur. are short for histogram, variance, skewness and kurtosis. The mean scores of the histogram are nearly the same. However, the histograms themselves with their statistics differ from each other. Images are from the AVA dataset (Murray, Marchesotti, and Perronnin 2012), which contains a list of photo IDs from www.dpchallenge.com.**

**The histogram of numbers of images located in different intervals of the mean and standard deviation of the AVA dataset (Murray, Marchesotti, and Perronnin 2012).**

**Left: Distributions of skewness of score distributions, for images with different mean scores. The red crosses are the outliers. The skewness tends to decrease from positive to negative with the mean score increasing. Right: Distributions of kurtosis of score distributions, for images with different mean scores. **

**Distributions of mean and median of score distributions, for images with different skewness scores. The divergences between the mean and the median distributions tends to increase with the distance between the skewness values and 0, which is the skewness of the symmetrical normal distribution. **

**Predicted score histograms by the above loss functions. The numbers above each histograms are their mean scores. The first column is the images. The 2nd column is the human rating distributions (GT: Ground Truth). The 3rd and the 4th columns are the results predicted by our proposed RS-CJS and CJS based CNN, respectively. The other columns are the predicted results of other loss functions. Ourresults are more similar to the ground truth of human rat- ings than others. Images are from the AVA dataset (Murray,Marchesotti, and Perronnin 2012), which contains a list of photo IDs from www.dpchallenge.com. **

**Paper**

**BibTeX**

@article{DBLP:journals/corr/abs-1708-07089, author = {Xin Jin and Le Wu and Chenggen Song and Xiaodong Li and Geng Zhao and Siyu Chen and Jingying Chi and Siwei Peng and Shiming Ge}, title = {Predicting Aesthetic Score Distribution through Cumulative Jensen-Shannon Divergence}, journal = {CoRR}, volume = {abs/1708.07089}, year = {2017}, url = {http://arxiv.org/abs/1708.07089}, archivePrefix = {arXiv}, eprint = {1708.07089}, timestamp = {Tue, 05 Sep 2017 10:03:46 +0200}, biburl = {http://dblp.org/rec/bib/journals/corr/abs-1708-07089}, bibsource = {dblp computer science bibliography, http://dblp.org} }