Lovely lab walk at Richmond Park with bambies and dragon lillies!

Using data from the ECOLANG corpus of naturalistic interactions between adult-child (aged 3-4), and adult-adult, we show that surprisal predicts speakers’ multimodal adjustments and that these effects are modulated by whether the comprehender is a child or an adult.

We ask whether speakers dynamically adjust their intonation and produce iconic gestures to increase communicative success, and decrease cognitive effort, by testing if they adjust depending on words predictability (surprisal, computed for Child-and Adult-Directed Language).

BUT, they showed larger effects for representational gestures and mouth movements

Overall L2 comprehenders show smaller modulation of N400 than L1 comprehenders for all cues

We looked at how the N400 elicited by more/less surprising words is modulated by: representational and beat gestures, prosody and mouth movements (same as in Zhang et al., 2021).

Huge thank you to participants to the 1st Multimodal Language Corpora Workshop! Looking forward to the 2nd in Marseille! @Adinseg

So proud of our collaboration with @MossResearch! here is our little group: @AnnaKrason, @LaurelBuxbaum, @IsobelAC_SLT

3. @Ed_Donnellan @a_jordanbarros et al: We investigate impact of multimodal cues in caregivers' communication to children (3-4, ECOLANG corpus) on immediate learning and vocabulary growth. Prosody and points impact both measures. escholarship.org/uc/item/6km…

2. @YumengW_Marine @Ed_Donnellan et al.: We analyse the timing between iconic gestures and lexical affiliates in caregivers communication with 3-4 years old children (ECOLANG corpus). Gestures precede early acquired BUT follow later acquired words. escholarship.org/uc/item/4fz…

1. @foobarinb @YeZhang_Claudia @vigliocco_g Does surprisal from n-gram or transformer models better predicts EEG (N400) responses for audio and audiovisual naturalistic language ? GPT2 is better for audio BUT n-grams are better for audio-visual. Link: escholarship.org/uc/item/8k0…