Speech Emotion Recognition in Conversations Using Graph Convolutional Networks

Simple item page
dc.contributor.advisorJenkin, Michael R.
dc.contributor.authorChandola, Deeksha
dc.date.accessioned2024-03-18T17:49:42Z
dc.date.available2024-03-18T17:49:42Z
dc.date.issued2024-03-16
dc.date.updated2024-03-16T10:56:25Z
dc.degree.disciplineElectrical and Computer Engineering
dc.degree.levelMaster's
dc.degree.nameMASc - Master of Applied Science
dc.description.abstractSpeech emotion recognition (SER) is the task of automatically recognizing emotions expressed in spoken language. Current approaches focus on analyzing isolated speech segments to identify a speaker’s emotional state. That being said, models based on text-based emotion recognition methods are considering conversational context and are moving towards emotion recognition in conversation (ERC). With the availability of multimodal datasets, ERC can be extended to non-text modalities as well. Building on these advances, in this thesis, we propose SERC-GCN, a method for speech emotion recognition in conversation (SERC) that predicts a speaker’s emotional state by incorporating conversational context, specifically speaker interactions, and temporal dependencies between utterances. SERC-GCN is a two-stage method. In the first stage, emotional features of utterance-level speech signals are extracted using a graph-based neural network. Here each individual speech utterance is transformed into a cyclic graph. These graphs are then processed by a two layered GCN architecture followed by a pooling layer to extract utterance-specific emotional features. In the second stage, these features are used to form conversation graphs that are used to train a graph convolutional network to perform SERC. We empirically evaluate the effectiveness of SERC-GCN on two benchmark dataset; IEMOCAP and MELD. Results show that SERC-GCN outperforms existing baseline approaches on these datasets.
dc.identifier.urihttps://hdl.handle.net/10315/41852
dc.languageen
dc.rightsAuthor owns copyright, except where explicitly noted. Please contact the author directly with licensing requests.
dc.subjectComputer science
dc.subjectComputer engineering
dc.subjectPsychology
dc.subject.keywordsSpeech emotion recognition in conversation
dc.subject.keywordsHuman-computer interaction
dc.subject.keywordsGraph convolutional network
dc.subject.keywordsEmotion recognition in conversation (ERC)
dc.subject.keywordsMultimodal analysis
dc.titleSpeech Emotion Recognition in Conversations Using Graph Convolutional Networks
dc.typeElectronic Thesis or Dissertation

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Chandola_Deeksha_2024_Masters.pdf
Size:
6.07 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
license.txt
Size:
1.87 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
YorkU_ETDlicense.txt
Size:
3.39 KB
Format:
Plain Text
Description:
Download

Collections

Electrical and Computer Engineering

All items in the YorkSpace institutional repository are protected by copyright, with all rights reserved except where explicitly noted.