کنفرانس مهندسی برق ایران

صفحه اصلی / سی و یکمین کنفرانس بین المللی مهندسی برق

Transformer-Based Unsupervised Image Registration using SSIM and Homography Loss for Steady Camera and Aerial Videos

نویسندگان :

Golnoosh Abdollahinejad¹ Matin Hashemi²

1- دانشگاه صنعتی شریف 2- دانشگاه صنعتی شریف

کلمات کلیدی :

Computer Vision،Image Registration،Transformer Network،Deep Learning

چکیده :

Image registration is an essential and initial block in the pipeline of computer vision tasks and systems. It is defined as the process of transforming a moving image into a target image with minimum difference when get aligned. Unlike previous work for general-purpose datasets, aerial images suffer from mechanical shakes which leads to deformed distortion that is similar to medical volumetric image registration task. Inspired by medical approaches, we use a Transformer-based network to have a semi-unlimited receptive field Swin block, to produce a general output for each pixel named flow matrix. Flow matrix is utilized instead of regressing parameters of transformation matrix with a fixed degree of freedom that can't handle the structural difference between images. This leads to introducing of a new loss function based on the Structural Similarity Index Measure (SSIM) and embedding Homography transformation as a regularization term. The combination of a generalized-designed network and loss function based on problem definition significantly enhanced results.