基于表情及姿态融合的情绪识别

Emotion recognition based on fusion of expression and posture

摘要:

情绪识别旨在使计算机拥有能够感知和分析人类情绪和意图的能力，从而在娱乐、教育、医疗、公共安全等领域发挥作用. 与直观的面部表情相比，身体姿态在情绪识别方面的作用总是被低估. 针对公共空间个体人脸分辨率较低、表情识别精度不高的问题，提出了融合面部表情和身体姿态的情绪识别方法. 首先，对视频数据进行预处理获得表情通道和姿态通道的输入序列；然后，使用深度学习的方法分别提取表情和姿态的情绪特征；最后，在决策层进行融合和分类. 构建了基于视频的公共空间个体情绪数据集(SCU-FABE)，在此基础上，结合姿态情绪识别数据增强，实现了公共空间个体情绪的有效识别. 实验结果表明，表情和姿态情绪识别取得了94.698%和88.024%的平均识别率；融合情绪识别平均识别率为95.766%，有效融合了面部表情和身体姿态表达的情绪信息，在真实场景视频数据中具有良好的泛化能力和适用性.

Abstract:

Research shows that the role of body posture in emotion recognition is always underestimated. Aiming at the problems of low face resolution and low expression recognition accuracy in public space, an emotion recognition method based on facial expression and body posture is proposed. Firstly, the video data is preprocessed to obtain the input sequence of expression channel and posture channel; then, the emotional features of expression and posture are extracted by deep learning method; finally, fusion and classification are carried out in decision level. The emotion dataset (SCU-FABE) based on public space video is constructed. On this basis, combined with posture emotion recognition data enhancement, the effective recognition of individual emotions in public space is realized. The experimental results show that the recognition rate of expression and posture is 94.698% and 88.024%; fusion emotion recognition rate is 95.766%. The proposed method effectively integrates emotional information expressed by facial expression and body posture, and has good generalization ability and applicability in real scene video data.