CausalityExperiment

CausalityExperiment

This is a psycology experiment to see the causal relationship between emotions when two subjects interact. The experiment shows the analysis of situations and emotions established by one subject that brings the other subject to mimic the emotions.

Process data

Change all paths in config and python files. Location of the videos are also required.

1. Clip video: The video of subject 1 and 2 are not synchronised. Therefore, 00_clip_video.py is used to clip the video at start and stop when clap is heard. This is in this case optional as offset values are provided.

2. Extract Action Units: Install OpenFace 2.0. This is used to extract AU (Action Units) from each frames of the videos. To extract and save the files, open terminal and go to the directory where OpenFace is installed. Then execute:

./bin/FeatureExtraction -f "/home/path_to_video_file/clipped_video_file.mp4"

The extracted data will be stored in the OpenFace directory inside the folder processed. We are interested in the .csv file.

3. Process offset: For this experiment the start and end in offset.csv file is extracted using 02_process_offset.py to slice the data.

4. Process data: This offset values are used in 03_process_data.py to complete the data processing.

Extract features

5. Get expressions: With 04_get_expression.py, the expressions are extracted from the processed data based on the combination of AU activation listed below.

• HappinessUpper: AU06
• HappinessLower: AU25, AU12
• SurpriseUpper: AU01, AU05, AU02
• SurpriseLower: AU26
• AngerUpper: AU04, AU05, AU07
• AngerLower: AU17, AU23
• DisgustLower: AU09, AU25, AU10
• FearUpper: AU01, AU04, AU05, AU02
• FearLower: AU25, AU20
• SadnessUpper: AU04, AU01
• SadnessLower: AU15, AU17

Analysis and results

• Wicoxon test: The 05_wilcoxon.py firstly normalizes the data. The test shows the percentage of each emotion detected throughout the experiment. The analysis also compares the amount of emotion activation in objective(o), respectful(a) and contempt(g) cases.

• Causality test: Firstly, the 07_causality.py calculates the relevant intervals between two pairs. High correlation means high relevance. Then Granger Causality is used to find the direction of influence. Finally, the causality_results.py is used to summarize the results.

For running Causality test on all pairs, utilize slurm if available. The run_slurm.py and slurm.sh files could be used for this.

References

Read the paper here https://ieeexplore.ieee.org/document/9425581