کنفرانس مهندسی برق ایران

صفحه اصلی / سی و دومین کنفرانس بین المللی مهندسی برق

High-Resolution Remote Sensing Image Captioning Based on Structured Attention and SAM Network

نویسندگان :

Yassin Riyazi¹ Seyyed Mostafa Sadjadi² Abbas Zohrevand³ Reshad Hosseini⁴

1- دانشگاه تهران 2- دانشگاه تهران 3- دانشگاه تهران 4- دانشگاه تهران

کلمات کلیدی :

image captioning،image segmentation،remote sensing image،structured attention

چکیده :

Due to its broad applications, remote sensing image captioning (RSIC) has gained popularity in recent years. However, it poses extra challenges for containing low-resolution images with highly structured semantic content. By incorporating image labeling and segmentation, this work expands on the RSIC framework developed by Zhao et al. [1]. The method presents a structured attention module that highlights important semantic components to maintain a geometric and structured shape. The quality and edge emphasis of UCM-captioned photographs is improved by upsampling them to 512×512 pixels. Using the Segment Anything Model (SAM) produces better image proposals, leading to higher accuracy than traditional techniques. A balanced output of large- and small-object masks is facilitated by SAM's promptability. The decoder can more easily learn a suitable statistical model using the model's spatial structure to provide an all-encompassing attention map. The effects of multiple hyperparameters, such as teacher forcing, the number of region proposals, and the effects of DSR and AVR loss factors, are investigated in this work. Overall, by combining image labeling and segmentation, this research improves remote sensing capabilities. It also shows how well the structured attention module and SAM work together to improve accuracy and consider different hyperparameter issues.