Emotion recognition based on fusion of expression and posture

Abstract:

Research shows that the role of body posture in emotion recognition is always underestimated. Aiming at the problems of low face resolution and low expression recognition accuracy in public space, an emotion recognition method based on facial expression and body posture is proposed. Firstly, the video data is preprocessed to obtain the input sequence of expression channel and posture channel; then, the emotional features of expression and posture are extracted by deep learning method; finally, fusion and classification are carried out in decision level. The emotion dataset (SCU-FABE) based on public space video is constructed. On this basis, combined with posture emotion recognition data enhancement, the effective recognition of individual emotions in public space is realized. The experimental results show that the recognition rate of expression and posture is 94.698% and 88.024%; fusion emotion recognition rate is 95.766%. The proposed method effectively integrates emotional information expressed by facial expression and body posture, and has good generalization ability and applicability in real scene video data.