Now that we have learned about some of the challenges of (human) annotation of images, as well as some of the major hurdles to the automation of the process, let’s explore and dig deeper into the area.

As well as discussing the challenges of image annotation in more detail, the following tutorial (http://www.micc.unifi.it/tagsurvey/) and related slides (http://www.micc.unifi.it/tagsurvey/downloads/mm15tut-full.pdf), provide a thorough overview of the wider issues of image retrieval (that is, searching for – and hopefully finding – relevant images), which is one of the main applications of automatic image classification.

Of the tutorial slides, Part 1 provides a good introduction to the area, with the other parts providing more detail (e.g. Part 4 focuses on machine learning techniques). Given the varied and technical nature of some of this material, the reader may wish to focus on some sections and skip others according to their interests and requirements. The tutorial material is structured as follows:

• Part 1: Introduction – provides an overview of images and their annotations as well as some issues of user behaviour in social media.
• Part 2: Taxonomy – highlights some of the choices involved in retrieval and refinement of images and their tags (or labels/annotations).
• Part 3: A new experimental protocol – presents some of the ways of evaluating the success of automated image annotation and some of the challenges.
• Part 4: Eleven key methods of media and learning – which focuses in particular on the machine learning aspects of image annotation.
• Part 5: Future directions – next steps in the research in this area.

A survey paper accompanies the tutorial and provides even more detail:

Xirong Li, Tiberio Uricchio, Lamberto Ballan, Marco Bertini, Cees G. M. Snoek, Alberto Del Bimbo, “Socializing the Semantic Gap: A Comparative Survey on Image Tag Assignment, Refinement and Retrieval”, ACM Computing Surveys (CSUR), Volume 49, Issue 1, 14:1-14:39, June 2016. (Available from arxiv.org,)