In the UK, the numbers of 16-year-olds studying a modern foreign language (French, German and Spanish) are low, as is foreign language proficiency when compared with the rest of Europe. We also know that UK learners make less use of foreign language multimedia outside of school than is the case in Europe. There, multimedia seems to have a beneficial impact on foreign language development.

Visages hideux collés contre la vitre. (Ugly faces pressed on the window.) Vitre [feminine noun] window Ila cassé une vitre. (He broke a window.)

Multimedia brings together text, audio, video, and/or images as, for example, in a video clip with captions or subtitles. Such materials have unique advantages for promoting foreign language learning. At a practical level, they are entertaining in nature and flexible for use in formal (classroom) settings by foreign language teachers and in informal (outside classroom) settings by learners themselves. At a theoretical level, they can deepen learners’ comprehension and learning by maximising the utilisation of their limited working memory capacity as verbal and nonverbal information can be processed simultaneously in different memory systems. In addition, exposure to multimedia requires learners to select, organise, and integrate information, which then tends to help them learn better.

There may be a vicious circle at work in the UK. That is, UK learners’ lower foreign language proficiency perhaps makes it harder for them to understand multimedia independently. Therefore, compared to their European peers, they are less likely to engage with such materials outside the classroom.

Moreover, we have limited understanding of what kinds of multimedia are most beneficial for what kind of learner, how they should be used, and what additional scaffolds might be needed for lower proficiency learners to benefit from them. This is a particularly important issue to explore as different learners may respond differently to the nonverbal cues (such as gestures and facial expressions) within multimedia. Greater sensibility to such cues normally leads to deeper understanding. These are all issues which my current study seeks to explore, to enhance understanding, for researchers and practitioners, of how multimedia use can improve foreign language comprehension and vocabulary growth.

The study involves a collaboration with both teachers and learners, to develop nine sets of foreign language multimedia learning materials (in French, German and Spanish). Each material will include a short bilingual subtitled movie clip (2–3 minutes long) alongside a few PowerPoint slides, designed and recorded by experienced foreign language teachers, teaching vocabulary which appeared in the clip.

These materials will be tried out and evaluated with 90 Year 7 learners in three schools. As learners view the materials individually on a laptop or tablet, their eye movements will be tracked by the built-in webcam through an online eye-tracking platform. This will show, for example, how much time they spend looking at visual or written elements of the materials. They will also complete a working memory test, a creative thinking abilities assessment task, and listening comprehension and vocabulary tests. This data will provide evidence to answer important questions: how the presence or absence of visual elements in multimedia materials affects listening comprehension, vocabulary learning, and attention allocation; and whether that differs for different kinds of learners.

We will organise a nationwide Creative Multimedia Materials “Mining” Competition for schools. Learners from schools across England will be invited and asked to find online the most interesting, creative, and linguistically accessible multimedia materials and identify what they learnt from them. A selection will be made from these materials, choosing those that incorporate an appropriate balance of visual and written input. These materials will be hosted on the project website and ranked by learners based on how engaging they found them. Learners submitting the five most highly-rated materials will receive a gift token as a prize.

The study is ongoing and currently at the stage of finalising the multimedia language learning materials. The next step will be piloting these materials to see how suitable they are for learners. We hope that the findings of the study will provide guidelines to help school modern foreign language teachers in English speaking countries who use multimedia for instruction to design their teaching content more effectively, with a particular focus on how to balance the use of different types of verbal and visual input to aid comprehension and learning.

Dr Anthony Zhang is a Lecturer in Education (Second Language Learning) at the University of Reading and the recipient of a University Research Fellowship 2022–23.