Predicting Aesthetic Score Distribution through Cumulative Jensen-Shannon Divergence

 Xin Jin1 , Le Wu1 , Xiaodong Li1 , Siyu Chen1 , Siwei Peng3 , Jingying Chi3 , Shiming Ge4 , ∗ , Chenggen Song2 , Geng Zhao1

1Department of Computer Sci. and Tech., Beijing Electronic Science and Technology Institute, Beijing, 100070, China

2Department of Info. Sec., Beijing Electronic Science and Technology Institute, Beijing, 100070, China

3College of Info. Sci. and Tech., Beijing University of Chemical Technology, Beijing 100029, China

4Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100093, China

∗Corresponding author email: geshiming@iie.ac.cn

Abstract

   Aesthetic quality prediction is a challenging task in the computer vision community because of the complex interplay with semantic contents and photographic technologies. Recent studies on the powerful deep learning based aesthetic quality assessment usually use a binary high-low label or a numerical score to represent the aesthetic quality. However the scalar representation cannot describe well the underlying varieties of the human perception of aesthetics. In this work,we propose to predict the aesthetic score distribution (i.e., a score distribution vector of the ordinal basic human ratings)using Deep Convolutional Neural Network (DCNN). Conventional DCNNs which aim to minimize the difference between the predicted scalar numbers or vectors and the ground truth cannot be directly used for the ordinal basic rating distribution. Thus, a novel CNN based on the Cumulative distribution with Jensen-Shannon divergence (CJS-CNN) is presented to predict the aesthetic score distribution of human ratings, with a new reliability-sensitive learning method based on the kurtosis of the score distribution, which eliminates the requirement of the original full data of human ratings (with- out normalization). Experimental results on large scale aesthetic dataset demonstrate the effectiveness of our introduced CJS-CNN in this task.

Gallery

aaai-1

Images with similar mean scores (i.e., around5). The rating distributions are approximated by the score histograms (1-10). The hist., var., skew. and kur. are short for histogram, variance, skewness and kurtosis. The mean scores of the histogram are nearly the same. However, the histograms themselves with their statistics differ from each other. Images are from the AVA dataset (Murray, Marchesotti, and Perronnin 2012), which contains a list of photo IDs from www.dpchallenge.com.

aaai-2

The histogram of numbers of images located in different intervals of the mean and standard deviation of the AVA dataset (Murray, Marchesotti, and Perronnin 2012).

aaai-4

Left: Distributions of skewness of score distributions, for images with different mean scores. The red crosses are the outliers. The skewness tends to decrease from positive to negative with the mean score increasing. Right: Distributions of kurtosis of score distributions, for images with different mean scores.

aaai-5

Distributions of mean and median of score distributions, for images with different skewness scores. The divergences between the mean and the median distributions tends to increase with the distance between the skewness values and 0, which is the skewness of the symmetrical normal distribution.

aaai-3

Predicted score histograms by the above loss functions. The numbers above each histograms are their mean scores. The first column is the images. The 2nd column is the human rating distributions (GT: Ground Truth). The 3rd and the 4th columns are the results predicted by our proposed RS-CJS and CJS based CNN, respectively. The other columns are the predicted results of other loss functions. Ourresults are more similar to the ground truth of human rat- ings than others. Images are from the AVA dataset (Murray,Marchesotti, and Perronnin 2012), which contains a list of photo IDs from www.dpchallenge.com.

Paper

pdf

Xin Jin, Le Wu, Chenggen Song, Xiaodong Li, Geng Zhao, Siyu Chen, Jingying Chi, Siwei Peng, Shiming Ge:Predicting Aesthetic Score Distribution through Cumulative Jensen-Shannon Divergence. CoRR abs/1708.07089 (2017)

BibTeX

@article{DBLP:journals/corr/abs-1708-07089,
  author    = {Xin Jin and
               Le Wu and
               Chenggen Song and
               Xiaodong Li and
               Geng Zhao and
               Siyu Chen and
               Jingying Chi and
               Siwei Peng and
               Shiming Ge},
  title     = {Predicting Aesthetic Score Distribution through Cumulative Jensen-Shannon
               Divergence},
  journal   = {CoRR},
  volume    = {abs/1708.07089},
  year      = {2017},
  url       = {http://arxiv.org/abs/1708.07089},
  archivePrefix = {arXiv},
  eprint    = {1708.07089},
  timestamp = {Tue, 05 Sep 2017 10:03:46 +0200},
  biburl    = {http://dblp.org/rec/bib/journals/corr/abs-1708-07089},
  bibsource = {dblp computer science bibliography, http://dblp.org}
}

发表评论

电子邮件地址不会被公开。 必填项已用*标注