Advancing Affective Computing: Emotion Recognition and Tracking Across Diverse Contexts (Varied Environments)

Author

Shao Liu, The Graduate Center, City University of New YorkFollow

Date of Degree

9-2024

Document Type

Dissertation

Degree Name

Ph.D.

Program

Computer Science

Advisor

Sos Agaian

Committee Members

Artyom Grigoryan

Hao Tang

Liang Zhao

Subject Categories

Artificial Intelligence and Robotics | Other Computer Sciences

Keywords

Affective Computing; Emotion Recognition; Emotion Tacking; Object Re-Identification

Abstract

Affective Computing (AC) is an interdisciplinary field that recognizes, interprets, and processes human emotions. Emotions are complex, involving consciousness, physical sensations, and behavioral expressions, and are significant in various domains like mental health, human-computer interaction, and social security. Real-world applications of AC include monitoring drivers’ emotional states to improve road safety and understanding the emotions expressed by artists in visual arts. Traditional methods relying on facial expressions often fall short due to the nuanced nature of emotions, which vary across individuals, cultures, and contexts. Accurate AC systems require sophisticated, multimodal models to handle these variations and external factors like noise and ambient light.

This PhD thesis aims to advance emotion recognition and tracking using deep-learning approaches, focusing on four main components: First, a three-dimensional emotion estimation model is developed to enhance the quantification and consensus of emotions in facial expressions for real scenarios and visual arts. Second, a novel dataset and framework are created to improve the comprehension and analysis of emotions conveyed in portrait arts. Third, an innovative multi-camera object tracking methodology is introduced to track multiple moving objects in various contexts. Fourth, using contexts and video data, a multimodal system is designed to recognize and track emotions in vehicles.

By addressing these components, the thesis seeks to overcome the limitations of conventional methods and provide robust solutions for emotion recognition and tracking in diverse and dynamic environments.

Recommended Citation

Liu, Shao, "Advancing Affective Computing: Emotion Recognition and Tracking Across Diverse Contexts (Varied Environments)" (2024). CUNY Academic Works.
https://academicworks.cuny.edu/gc_etds/6030