2024 · 15 citations · 25 references
ComputingComputer ScienceComputer VisionNatural Language ProcessingVisual RepresentationObject RecognitionVisual Question AnsweringCaptioningFine-tuningImage CaptioningIndexingTransformer ModelsVision Transformers
Transformer-based models have reshaped image captioning but grapple with issues like caption accuracy, particularly for complex visuals. Addressing these shortcomings is essential. Motivated by existing challenges, the Vision Transformer (ViT) as encoder and Generative Pretrained Transformer 2 (GPT-2) as decoder have been employed to enhance caption quality, utilizing the Seq2Seq framework and training on Flickr8k. This work introduced a novel ViT-GPT-2 image captioning model, evaluating it against benchmarks including Flickr8k. The model excels with BLEU-4 at 39.76 and METEOR at 52.30, bridging visual-textual gaps effectively. This research advances image captioning, offering practitioners an improved model for content indexing, accessibility, and human-computer interaction. ViT- GPT-2’s success underscores bridging semantic gaps in image captioning, with future work exploring diverse datasets and fine-tuning techniques for enhanced performance.
25
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text