ICMI 2018 Workshop on Modeling Cognitive Processes from Multimodal Data
Update 11/2021: Since this first installment of this workshop, it has developed into a workshop series at ACM ICMI on "on modelling socio-emotional and cognitive processes from multimodal data in the wild":
Multimodal signals allow us to gain insights about internal cognitive processes of a person, for example: Speech and gesture analysis yield cues about hesitations, knowledgeability, or alertness, eye tracking yields information about a person's focus of attention, task, or cognitive state, EEG yields information about a person's cognitive load or information appraisal. Capturing cognitive processes is an important research tool to understand human behavior as well as a crucial part of a user model to an adaptive interactive system such as a robot or a tutoring system. As cognitive processes are often multifaceted, a comprehensive model requires the combination of multiple complementary signals. In this workshop at the ACM International Conference on Multimodal Interfaces (ICMI) conference in Boulder, Colorado, USA, we will discuss the state-of-the-art in monitoring and modeling cognitive processes from signals. We welcome the presentation of evaluation studies, theoretical considerations, data corpora, and novel modeling approaches. Potential topics include but are not limited to:
- Multimodal workload and stress estimation
- Multimodal attention and memory modeling
- Multimodal engagement/frustration/confusion estimation
- Eye tracking in the wild
- Brain-Computer Interfaces and psychophysiological interfaces
- Cognitive modeling
- Experiment design for cognitive processes
- Cognition-adaptive human-computer interfaces
The workshop proceedings are now available in the ACM digital library!
Workshop program:
Every paper will be assigned a 15 minute presentation slot (+ some minutes for discussion).
| Time | Title | Presenter / Authors |
| 9:00-9:15 | Welcome Note | Felix Putze (University of Bremen) |
| Session 1: Cognition in Interaction | Chair: Felix Putze | |
| 9:15-9:35 | Predicting Group Satisfaction in Meeting Discussions | Catherine Lai (University of Edinburgh), Gabriel Murray (University of the Fraser Valley) |
| 9:35-9:55 | Multimodal Approach to Engagement and Disengagement Detection with Highly Imbalanced In-the-Wild Data | Dmitrii Fedotov (Ulm University); Olga Perepelkina (Neurodata Lab LLC)*; Eva Kazimirova (Neurodata Lab LLC); Maria Konstantinova (Neurodata Lab LLC); Wolfgang Minker (Ulm University) |
| 9:55-10:15 | Workload-Driven Modulation of Mixed-Reality Robot-Human Communication | Leanne Hirshfield (Syracuse University); Thomas E Williams (Colorado School of Mines); Natalie Sommer (Syracuse University); Trevor Grant (Syracuse University); Senem Velipasalar (Syracuse University) |
| 10:15-10:35 | Symptoms of Cognitive Load in Interactions with a Dialogue System | José David Lopes (Heriot Watt University); Helen Hastie (Heriot-Watt University); Katrin Lohan (Heriot-Watt University) |
| 10:35-11:00 | Coffee Break | |
| Session 2: Attention | Chair: Jutta Hild (Fraunhofer IOSB) | |
| 11:00-11:20 | Histogram of oriented velocities for eye movement detection | Wolfgang Fuhl (University of Tuebingen)*; Nora J Castner (University Tübingen); Enkelejda Kasneci (University of Tuebingen) |
| 11:20-11:40 | Estimating Cognitive Load in Passive and Active Tasks from Pupil Width and Eye Gaze Changes using Bayesian Surprise | Elena Wolf (Karlsruhe Institute of Technology); Manel Martinez (KIT)*; Alina Roitberg (KIT); Rainer Stiefelhagen (Karlsruhe Institute of Technology); Barbara Deml (Karlsruhe Institute of Technology) |
| 11:40-12:00 | Investigating Static and Sequential Models for Intervention-Free Selection Using Multimodal Data of EEG and Eye Tracking | Mazen Salous (University of Bremen)*; Felix Putze (University of Bremen); Tanja Schultz (University of Bremen); Jutta Hild (Fraunhofer IOSB); Jürgen Beyerer (Fraunhofer IOSB) |
| 12:00-13:00 | Lunch | |
| 13:00-13:20 | Overlooking: The nature of gaze behavior and anomaly detection in expert dentists | Nora J Castner (University Tübingen)*; Enkelejda Kasneci (University of Tuebingen) |
| 13:20-13:40 | Rule based learning for eye movement type detection | Wolfgang Fuhl (University of Tuebingen)*; Nora J Castner (University Tübingen); Enkelejda Kasneci (University of Tuebingen) |
| Session 3: Neural & Cognitive Modeling | Chair: Erin Solovey | |
| 13:40-14:00 | Integrating Non-Invasive Neuroimaging and Computer Log Data to Improve Understanding of Cognitive Processes | Leah Friedman (Drexel University), Ruixue Liu (Worcester Polytechnic Institute), Erin Walker (University of Pittsburgh), Erin T. Solovey (Worcester Polytechnic Institute) |
| Multimer: Validating Multimodal, Cognitive Data in the City | Arlene B Ducao (MIT); Ilias Koen (New York University); Zhiqi Guo (New York University) | |
| 14:20-14:45 | Coffee Break | |
| 14:45-15:05 | The Role of Emotion in Problem Solving: First Results from Observing Chess | Thomas Guntz (Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France )*; James L. Crowley (Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France ); Dominique Vaufreydaz (Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France); Raffaella Balzarini (Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France); Philippe Dessus (Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France) |
| 15:05-15:25 | Discovering Digital Representations for Remembered Episodes from Lifelog Data | Bernd Dudzik (Delft University of Technology); Joost Broekens (Delft University of Technology); Mark Neerincx (TU Delft); Jeffrey Olenick (Michigan State University); Chu-Hsiang Chang (Michigan State University); Steve W. J. Kozlowski (Michigan State University); Hayley Hung (TU Delft) |
| 15:25-15:45 | Multimodal Framework for Cognitive Task Performance Prediction from Body Postures, Facial Expressions and EEG Signal | Ashwin Ramesh Babu (University of Texas at Arlington)*; Akilesh Rajavenkatanarayanan (University of Texas at Arlington); James Robert Brady (University of Texas at Arlington); Fillia Makedon (University of Texas at Arlington) |
| 15:45-16:30 | Closing Discussion |
Organizers:
- Felix Putze, University of Bremen
- Jutta Hild, Fraunhofer IOSB
- Enkelejda Kasneci, University of Tübingen
- Akane Sano, MIT Media Lab/Cornell University
- Erin Solovey,Worcester Polytechnic Institute
- Tanja Schultz, University of Bremen
Important Dates:
- Submission deadline: August 5th 2018 (extended!)
- Notification of acceptance: September 9th, 2018
- Camera-ready manuscript due: September 16th, 2018 (extended, you will receive further information by email)
- Workshop date: October 16th, 2018
Find here the Call for Papers.
Submit your paper here.