Bringing Words to Life

The Gruffalo is a children’s story of a mouse walking through the "deep dark wood", outwitting potential predators along the way. With augmented reality (AR) apps becoming increasingly common, children now have the chance to become active characters, navigating in their own way through the forest. 

"It’s an experience that’s very distinct from reading the book or watching a video. They hear what’s in the forest, they see the light or lack of light from the first-person perspective," says Dr Lynde Tan, a senior lecturer in primary education at Western Sydney University. 

Tan is at the forefront of integrating emerging technologies such as AR into the teaching of primary literacy. In addition to reading, writing, speaking, and listening, Australia’s English curriculum recognises literacy as multiple ways of making meaning, such as using words, images, gestures and sounds. Teachers often use printed picture books, videos, and films to demonstrate the interplay between words and images. 

With the capacity to engage children through multiple senses, AR has the potential to enhance multimodal literacy learning for children in innovative ways.  

The educational benefits of using AR have been well explored for teaching science and mathematics. "It allows students to study things beyond what they can access in their everyday lives, like examining a planet as though you can hold it in your hands," says Tan.  

This led Tan to examine whether these benefits could also be applied to literacy teaching, particularly in primary schools.  

"While many associate AR primarily with recreational uses, such as the game Pokemon Go, its capability to interact with text using multiple senses — sight, sound, and touch — makes it particularly relevant to literacy," she says. "To equip young students for their futures, emerging technologies such as AR need to be integrated into curricula from the early learning stages, so that we can explore how all the educational benefits can be leveraged for meaningful, educational use."

Tan has started to co-design lessons with teachers so that the use of AR aligns with the Australian Curriculum for English and the intended learning outcomes.  

"For us, it was important that there was evidence-based research behind the use of AR. I had reservations about how we could integrate newer technologies meaningfully and strategically at first," explains Nicole Tainsh, the principal of Vaucluse Public School in Sydney. Tainsh collaborated with Tan when she was a deputy principal of another school with a large population of students for whom English is an additional language. 

"I was interested in using AR not as a small, fun activity on the side, but something that augments the learning," she adds. "At the same time, I thought AR could be an excellent way to foster children’s understanding of what a virtual world is, because there is overlay with the real world."

In the collaborating school’s Year 2 class, students learned about space. Teachers showed students books on the solar system and galaxy, illustrating the vastness of the cosmos. Later, students used an AR app, teleporting to the Moon as astronauts, which represented space differently than the book. The teachers guided the students to pay attention to the pitch, volume and pace of sound on the AR app, for example, when the rocket was launching to the Moon. The qualities of sound suggest how far the Moon is from Earth. The spatial mode is also examined when the students explored the Moon as an astronaut. 

The potential to accelerate vocabulary and reading comprehension also stood out for Tan. "Comprehension can be learned through actions, much like how a toddler acquires language through contextual cues. When students hear an instruction to 'get into the rover', they see an astronaut pointing to the rover. Nothing happens until they walk into it. They picked up new words through the tacit, AR-mediated experience," says Tan, who adds that they were very engaged in the experience.  

"When receiving instructions, the students were noticeably quiet. As soon as they began using the app outdoors, however, they became excited to be a character in the story," Tan says. "The teachers were also amazed. Using AR was like taking their students to a new world, and they forgot where they were. One teacher commented that being immersed in the half-real, half-virtual world facilitated deep conversations about the topic."  

The initiative impacted Tan’s practice in the long term, helping craft units of work with AR that were practical, aligned with the syllabus, and based on best teaching principles. In addition, Tan devised a pedagogical framework and evaluation criteria for AR apps so that teachers can select those that are practical and know how to use them effectively to cater toward teaching goals.  

"While apps evolve, I have the knowledge to determine if an app can be meaningfully integrated into lessons. The word is spreading among educators about its effectiveness and engagement potential," she says. "We want children to be creative and critical users of technology as they grow older; these lessons show children as active learners and how they use technology beyond passive consumption."

Acknowledgement
The AR project entitled Language and Literacy Learning and Teaching with Augmented Reality was funded by Primary English Teaching Association Australia.