کنفرانس مهندسی برق ایران

صفحه اصلی / سی و چهارمین کنفرانس بین المللی مهندسی برق

Towards Better Domain Generalization with Jacobian Norm Regularization

نویسندگان :

Abolfazl Ettehadieh¹ Mostafa Tavassolipour² Hamid Soltanian-Zadeh³

1- دانشگاه تهران، دانشکده مهندسی برق و کامپیوتر 2- دانشگاه تهران، دانشکده مهندسی برق و کامپیوتر 3- دانشگاه تهران، دانشکده مهندسی برق و کامپیوتر

کلمات کلیدی :

Domain Generalization،Jacobian Regularization،Flat Minima،Robustness

چکیده :

Domain generalization (DG) aims to train models that perform well on unseen target domains despite distributional shifts from the source domains. In this work, we propose a regularization approach based on estimating the Jacobian norm to improve model generalization by reducing sensitivity to input variations. Prior studies have demonstrated that the Jacobian norm correlates with generalization performance, as it measures model sensitivity to perturbations, and that it is linked to flatter minima on the loss landscape. Building on these insights, we approximate the Frobenius norm of the Jacobian using the Hutchinson trace estimator to avoid high computational costs and incorporate it as a penalty term in the empirical risk minimization (ERM) loss; we further combine this method with SWAD,anensemble technique that promotes flat minima through weight averaging, to enhance its effectiveness. We evaluate our approach using the DomainBed protocol on four benchmark datasets: VLCS, PACS, Office-Home, and TerraIncognita. Results show consistent improvements over the ERM baseline, such as a 1.5% increase in mean accuracy on VLCS (reaching 79.0%). Additionally, visualizations of attention maps indicate that our method extracts more domain-invariant features, focusing on category-related elements rather than backgrounds. Overall, this Jacobian-based regularization offers a simple yet effective way to boost DG, supported by its connections to robustness and generalization from previous research. Source code is available at https://github.com/Ettehadieh/JR DG