While convolutional neural networks have become a transformative tool for many image analysis tasks, it is still common in literature to describe these deep learning approaches as ‘black boxes.’
To address these concerns, there have been substantial efforts to understand and visualize the features of classification networks, which take in an image and predict a class label.
However, much less work has been done to visualize and understand similarity networks, which learn an embedding that maps similar examples to nearby vectors in feature space and dissimilar examples to be far apart.
This talk, given by Abby Stylianou Ph.D., will focus on efforts to improve these sorts of embedding networks, and visualize and understand them, with a focus on networks trained on hotel recognition. The task of identifying what hotel is seen in a photograph taken in a hotel. Hotel recognition is a challenging recognition problem due to the properties of hotel rooms, including low visual similarity between rooms in the same hotel and high visual similarity between rooms in different hotels, particularly those from the same chain.
Building accurate approaches for hotel recognition is important to investigations of human trafficking. Images of human trafficking victims are often shared by traffickers among criminal networks and posted in online advertisements, and these images are often taken in hotels. Using hotel recognition approaches to determine the hotel a victim was photographed in can assist in investigations and prosecutions of human traffickers.
Stylianou’s research has focused on building very large datasets of images and their metadata, as well as the deep learning approaches to digest and understand those datasets for a variety of tasks. These tasks include performing hotel-specific image retrieval in order to locate victims of sex trafficking who have been photographed in hotels, making measurements of the natural environment in time-lapse imagery to understand climate change, and observing how individuals interact with the world around them in outdoor webcam images to support better design of the built environment.
She is particularly interested in how motivated communities of non-experts can participate in the collection of these datasets and what artifacts and challenges are introduced by capturing data from such diverse populations.
Stylianou additionally works to understand how computer vision and machine learning solutions to complex geometric and image retrieval problems can be made accessible and understandable to non-technical individuals, such as citizen journalists and law enforcement investigators, through software interfaces designed specifically with non-experts in mind.