PUC Chile team at Caption Prediction: ResNet visualencoding and caption classification with Paramet

RL1, Publisher: CLEF2021 Working Notes, CEUR Workshop Proceedings, Link>

AUTHORS

Vicente Castro, Pablo Pino, Denis Parra, Hans Lobel

ABSTRACT

This article describes PUC Chile team’s participation in the Caption Prediction task of ImageCLEFmedical challenge 2021, which resulted in the team winning this task. We first show how a very simple approach based on statistical analysis of captions, without relying on images, results in a competitive baseline score. Then, we describe how to improve the performance of this preliminary submission by encoding the medical images with a ResNet CNN, pre-trained on ImageNet and later fine-tuned with the challenge dataset. Afterwards, we use this visual encoding as the input for a multi-label classification approach for caption prediction. W