Deep Image Aesthetics Classification using Inception Modules and Fine-tuning Connected Layer

Xin Jin1,∗ , Jingying Chi2 , Siwei Peng2 , Yulu Tian1 , Chaochen Ye1 and Xiaodong Li1,∗

1Beijing Electronic Science and Technology Institute, Beijing 100070, China

2Beijing University of Chemical and Technology, Beijing 100029, China

Abstract

   In this paper we investigate the image aesthetics classification problem, aka, automatically classifying an image into low or high aesthetic quality, which is quite a challenging problem beyond image recognition. Deep convolutional neural network (DCNN) methods have recently shown promising results for image aesthetics assessment. Currently, a powerful inception module is proposed which shows very high performance in object classification. However, the inception module has not been taken into consideration for the image aesthetics assessment problem. In this paper, we propose a novel DCNN structure codenamed ILGNet for image aesthetics classification, which introduces the Inception module and connects intermediate Local layers to the Global layer for the output. Besides, we use a pre-trained image classification CNN called GoogLeNet on the ImageNet dataset and fine tune our connected local and global layer on the large scale aesthetics assessment AVA dataset [1]. The experimental results show that the proposed ILGNet outperforms the state of the art results in image aesthetics assessment in the AVA benchmark.

wcsp2016-classification

wcsp2016-accuracy

 

 

 

 

 

 

 

 

The architecture of the proposed ILGNet: Inception with connected Local and Global layers. We use one pre-treatment layer and three inception layers. The first two inception layers extract local features and the last one extracts global features. Recent work shows value in directly connecting intermediate layers to the output. Thus, we connect the two layers of local features to the layer of global features to form a concat layer of 1024 dimension to a full connected layer. The output layer is 1 dimension which indicate low or high aesthetic quality. The network is 13 layers deep when counting only layers with parameters (or 17 layers if we also count pooling). The labels (1)-(7) are used for the visualization in section IV.

20170116214114

 

 

 

 

 

 

 

 

 

The histogram/distribution of the mean scores and the number ofvotes per image in the AVA dataset.

20170116214137

 

 

 

 

 

 

 

The visualization results of the the weights of the first three convolutional layers. The labels of (1), (2), (3) correspond to the same labels in the third picture.

20170116214709

 

 

 

 

 

The visualization results of the the weights of the features extracted by our ILGNet in important layers for images with high (top) and low (bottom) labels. The labels of (1)-(7) correspond to the same labels in the third picture.

20170116214158

 

 

 

 

 

Pdf       GitHub

The Trained Models

The size of the trained model is above 500MB.

You can download them from the BaiduYun cloud disk or Google Drive:

BaiduYun Links:

ILGnet-AVA1.caffemodel

ILGnet-AVA2.caffemodel

Google Drive Links:

ILGnet-AVA1.caffemodel

ILGnet-AVA2.caffemodel

8 thoughts on “Deep Image Aesthetics Classification using Inception Modules and Fine-tuning Connected Layer

  • 2017年1月15日 at pm10:37
    Permalink

    This is a good paper.

    Reply
  • 2017年5月22日 at pm11:43
    Permalink

    After I initially left a comment I appear to have clicked on the -Notify me when new comments are added- checkbox and now each time a comment is added I receive 4 emails with the exact same comment. There has to be an easy method you can remove me from that service? Thanks!

    Reply
  • 2017年5月23日 at pm11:59
    Permalink

    Hi, just wanted to say, I liked this blog post. It was helpful. Keep on posting!

    Reply
  • 2017年5月25日 at am12:29
    Permalink

    Having read this I thought it was very enlightening. I appreciate you spending some time and energy to put this content together. I once again find myself personally spending way too much time both reading and commenting. But so what, it was still worthwhile!

    Reply
  • 2017年5月26日 at am2:08
    Permalink

    Its like you learn my mind! You appear to grasp a lot about this, like you wrote the ebook in it or something. I think that you can do with a few % to power the message home a bit, but other than that, this is great blog. An excellent read. I’ll certainly be back.

    Reply
  • 2017年5月27日 at pm6:26
    Permalink

    Hello, i believe that i noticed you visited my site thus i got here to go back the choose?.I’m trying to find issues to enhance my website!I guess its good enough to use some of your ideas!!

    Reply
  • 2017年5月27日 at pm6:26
    Permalink

    Hi there, just wanted to say, I enjoyed this blog post. It was helpful. Keep on posting!

    Reply
  • 2017年7月7日 at am1:01
    Permalink

    Abuja — Resuming from break, yesterday, the Senate immediately
    went into a standoff with the presidency, vowing to stand down all requests for confirmation into executive positions in the Federal Government.

    The Senate’s stance was upon overflow of outrage at the
    challenge from Acting President Yemi Osinbajo to the Senate’s capacity
    to confirm certain nominees of the executive into federal
    bodies.

    Senators across partisan lines were particularly miffed by the continued stay in office of Mr.
    Ibrahim Magu as chairman of the Economic and Financial Crimes Commission,
    EFCC, despite his rejection by the Senate on two occasions.

    In their resolution, the Senate asked the Acting President to apologise for his challenge to the Senate’s power to confirm persons
    into certain executive bodies.

    The Senate had at the resumption of plenary yesterday, entered into a closed session which lasted for almost an hour.
    A source at the meeting said senators were not happy over what they termed, “flagrant refusal by the executive to implement resolutions passed by the Senate.”

    The Acting President had last April in an interview said:
    “It is up to the Senate to make their judgment, and it is up to us to say what we want to do. If our candidate is rejected, we can re-present him. No law says we can’t re-present him. And again, there is the other argument, whether or not we need to present him for confirmation and that’s a compelling argument from Femi Falana,” he said.

    Yesterday’s resolution followed the reading of a letter from Osinbajo to the Senate requesting it to confirm Lanre Gbajabiamila as the substantive Director-General of the National Lottery Commission (NLC).

    Osinbajo’s letter which was dated June 12, 2017, was entitled, ” Request for confirmation of the appointment of Mr. Lanre Gbajabiamila as Director- General, National Lottery Regulatory Commission.”

    The letter read: “In compliance with the provisions of section 8 (1A) of the National Lottery Act 2005, I write to forward the name of Mr. Lanre Gbajabiamila as the nominee for the position of Director General, National Lottery Regulatory Commission for the consideration and appointment of the Senate.

    “If confirmed, Lanre Gbajabiamila will serve for an initial period of four
    years and may be reappointed for a further term of four years
    and no more.

    “The curriculum vitae of Lanre Gbajabiamila is attached for your perusal. It is, therefore, my hope that the Distinguished Senate of the Federal Republic of Nigeria do consider and confirm the appointment of the nominee in the usual expeditious manner.”

    Once the letter was read by the President of the Senate, Senator Bukola Saraki,
    Senator Ahmed Sani, APC, Zamfara West, upon Order 14
    of the Senate Standing Rules, argued that since the Acting President
    had already concluded that the Senate lacked the
    powers to confirm nominees, there was no need to acknowledge any letter from the executive
    on issues concerning confirmation.

    According to him, the issue must be clarified by the Senate before going ahead with
    the consideration of further requests from the presidency.

    Nigerians Attack Senate Over Rejection of Nominees

    “I want to draw the attention of the Senate to a statement credited to the Acting President, Professor Yemi Osinbajo that the Senate does not have the powers to confirm. I was surprised to see the Senate President read a letter from the same Acting President who said the Senate does not have the powers to confirm a nominee from the executive.

    “I am raising this point of order to draw the attention of the Senate to this issue and urge that we suspend any further action on confirmation of any nominee until the issue is
    resolved.”

    Supporting him, Senator George Sekibo, PDP, Rivers East took a swipe at the presidency over its continued refusal to comply with the resolutions of the Senate as he stressed that it was a clear attempt by the presidency to usurp the powers and roles of the apex legislative body.

    Senator Sekibo, who quoted relevant sections of the constitution to buttress his argument, said: “What is happening shows to Nigerians that they (executive) are either trying to quiet the National Assembly or trying to take
    over our responsibilities. From the constitution, it is very clear that one of the functions of the Senate is confirmation of
    appointments by the President.

    Other senators who spoke in the same line included Senators Isah Hamman Misau,
    APC, Bauchi Central, who accused the Acting President of using double
    standards on the Senate’s confirmation powers; Senator Matthew Urhoghide,
    PDP, Edo South, who urged the executive to approach the Supreme
    Court if it wanted to contest the powers of the Senate and Senator Samuel Anyanwu, PDP, Imo East, who issued a 48-hour ultimatum to Saraki
    to do something about the controversial statement by the Acting President.

    “We must take a stand. If the leadership does not do anything about this issue within 48 hours, we will move against the leadership. I want to say that anybody who is invited by the EFCC should not go because they are not standing on any law.”

    Also contributing, Senator Dino Melaye, APC, Kogi West
    said: “It is time for the Senate to tell the Executive Arm of government that they must stop approbating and reprobating. This republic that our fathers fought for should not be allowed to be completely destroyed. Not in our time, God forbid.

    “Magu came for a job interview and failed, and he was rejected.
    As we speak, Magu is still parading himself as the chairman of EFCC.
    The same executive now writes a letter that we should confirm
    someone who went to resume office without confirmation, and he was chased out of office.
    Now that name has been brought before us.

    “The integrity of the Senate is at stake. This Senate cannot be disregarded and insulted; our authority to confirm cannot be eroded. I am moving a motion that it becomes abominable to read any other confirmation report in this chamber until they act on our position. Those who have failed should go back, Magu is not the last Nigerian Angel.

    He can serve this country in another capacity.”

    Senate Chief Whip, Senator Olusola Adeyeye, APC, Osun Central, insisted that the executive
    must obey the provisions of the constitution and expressed his disappointment, saying that though he voted for Magu, he had to abide by the overriding vote of the Senate to reject him.

    “Every public servant is bound to obey every law of the Republic. Nobody, not the President, Vice or the Senate President has the right to disobey our laws. Whoever has problems with our laws should go to court and declare them null and void. Until that is done, every law of the land must be respected.

    “I voted yes for Magu. But the Senate of the Federal Republic
    of Nigeria voted no. I stand with the Nigerian Senate.
    I choose a strong institution. We should not allow
    our institutions to be weakened.”

    According to Adeyeye, if there is any law passed by the National Assembly, signed by the President and gazetted, no one has the permission to dance around that law, adding, “Whoever
    has a problem with any law should go to court, until a court declares it null and void it remains the law
    of the land. Anyone who does not respect it is breaking the law of
    the land.”

    On his part, the Deputy Senate President, Senator Ike Ekweremadu, who noted that the executive must obey the contents of the Constitution, said: “Let me make it clear that this has nothing to do with the Director-General of Lottery Commission or Ibrahim Magu, but about our constitution. One of the features of the Senate is the power of confirmation. The framers of the Constitution gave the
    power to the Senate.

    “If the law says that an appointee requires confirmation, we must obey what the law says. The executive must follow the provisions of the constitution.”

    A move by Senator Enyinnaya Abaribe, PDP, Abia South to draw
    attention to the absence of the Acting President in the country for two
    days running which he said had left a power vacuum was hushed down on the intervention of the Senate
    President.

    For about five minutes, from 12.25 to 12.30 Southwest senators clubbing together with some Northern senators stood against
    Abaribe’s interjection.

    Senator Kabiru Marafa, APC, Zamfara Central, countered Abaribe and
    was upheld by the Senate President.

    Interestingly, of the 11 senators that spoke yesterday,
    six were of the APC, while the other five were of
    the PDP.

    Upholding the point of order raised by Senator Yerima, the President of the Senate
    warned that the Senate must show serious action on the matter for once and for all and
    put it behind them.

    Senator Saraki said: “This is an issue that we have once and for all address; we cannot pass laws and see that the laws are not being obeyed. This resolution must be obeyed by the Acting President otherwise we will take appropriate actions and resolutions.

    “It is very clear these resolutions as passed must be acted
    upon by the Acting President, and ensure that we continue to respect our democracy, our laws, and constitution. It is not for
    us to choose which laws we obey and which laws we don’t obey.
    That is not the way any civilised, modern society
    work.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *