کنفرانس مهندسی برق ایران

صفحه اصلی / سی و یکمین کنفرانس بین المللی مهندسی برق

Vision Transformer and Parallel Convolutional Neural Network for Speech Emotion Recognition

نویسندگان :

Saber Hashemi¹ Mohammad Asgari²

1- دانشگاه صداوسیما 2- دانشگاه صدا و سیما

کلمات کلیدی :

speech emotion recognition،vision transformer،convolutional neural network،attention mechanism

چکیده :

Vision transformer (ViT) is a new approach for image processing tasks. The vision transformer splits the image into patches and converts it into a sequence of vectors. This sequence is suitable for the transformer structure. This paper uses the ViT method for speech emotion recognition. Unlike ViT, which splits the image into square patches, we use time frames as patches. Alongside using the frame-based ViT to benefit from its ability to learn global features, we are using a convolutional neural network. The convolutional neural network extracts local features and focuses on the two-dimensional structure of the input. Mel-Frequency Cepstral Coefficients extracted from audio files are used as input for the proposed neural network. Using this model in the RAVDESS dataset, we achieved an unweighted accuracy of 79.2%.