Awardee: Annie Brancky
Semester of Award: Spring 2019
Materials Awarded: A 360 camera, a 360 microphone, and a pair of VR goggles.
Project Description:
I. What did you hope to learn or achieve with the hardware and/or software you received?
Research shows that language learning works best in immersive and interactive environments, which we strive to establish in all of our language classrooms in FFS. However, given the obvious limitations of this aim, and given the growth of new technologies that allow for remote, virtual and augmented realities, I wanted to explore how we can enhance our immersive language-learning environment right here on campus or from any location for that matter. This is both a pedagogical priority and a question of access, since not all of our students can study in French-speaking locations during their junior year for a variety of reasons, and since the pandemic has strictly limited our ability to travel.
While there are a number of private educational and gaming companies that have been experimenting with different products that can use VR technology or 360 cameras for language learning (e.g. Mondly, Immerseme), there are also pedagogical limitations among commercial products such as these: they usually address the needs of beginning language learners, their situations are very functional, they tend to lack cultural diversity, they are not necessarily developed by experienced pedagogues.
In the academic domain, faculty at Grinnell College have been working on immersive VR experiences for language learning and cultural awareness. They are developing open-source VR games that teach about sustainability. Their model has a language-learning component, but has additional environmental protection content, as well as game design training and development, as the idea is for the students to create the games (https://gciel.sites.grinnell.edu/gciel-projects/).
After much discussion with Academic Computing, we decided that given the resources available to us now, rather than develop VR environments that would include gaming knowledge, programming, voice recognition technology, mic-ed headsets, etc., we should start simpler, experiment, and see what avenues we might want to pursue.
We decided to use a 360 camera to simulate an immersive, French-language “dinner” with friends or a host family, where the viewing student would have the illusion of being among a group of people, following a conversation in French at its natural pace and with natural language structures, in order to develop listening comprehension and cultural competency. Students would watch the 360 video with VR glasses and follow the conversation by looking towards the person speaking to pick up on gestures, facial expressions, and to be aided in comprehension by watching their mouths move. I would write listening comprehension and cultural competency questions related to the video content to help guide student viewing.
How is this different from watching a video?
Because of the immersive nature of 360 video, this VR scenario has several advantages over listening comprehension through audio recordings or even film.
- Kinesthetic: the student must move to follow the conversation, to look at what’s being pointed at, to find the exit sign in the metro, etc.
- Immersive: the student is fully immersed in a French environment
- Distraction-free: you can’t be texting or looking at Instagram while doing this
- We can craft the topics and scenarios to relate to our course
II. Were you able to put it to the use that you had planned? Please describe.
In the fall of 2019, with the help of ACS, I gathered 7 French-speaking students on campus in the FFS lounge and had them sit around a table, where we put the 360 camera and microphone. I asked the students a series of questions (some open-ended, some related to FFS 210 course materials) and encouraged them to speak freely, interrupt each other, use familiar or technical language, etc. The students involved were not actors, did not all know one other, and felt a little shy in front of the camera. They respected one another and listened to one another and spoke one at a time. So while the video is not as dynamic as we had hoped it would be, the content could nevertheless have applications.
Additionally, over the 2019-2020 winter break, Theo Mation, our Teaching Assistant, took the 360 camera to France and Belgium and filmed several short scenes in public spaces (in a café and in Brussels city center), to see what kinds of videos might be suitable for classroom use.
Unfortunately, very soon after the videos were processed, the planning, organization and implementation of this project was interrupted by the COVID-19 shutdown in March of 2020. While this type of pedagogical tool would potentially be very helpful in the sudden shift to online and hybrid learning, the videos as they are now are not quite ready for effective implementation for three reasons. One, while the videos we were able to record were very interesting and useful tests, they are not ideal for classroom use, due to either a lack of pedagogical content focus or due to poor sound quality. Two, because of scheduling issues related to the pandemic, I did not teach FFS 210 this year, which was the course in which I had intended to pilot these videos. Three, preparing the existing videos for use in the classroom would require a serious time commitment to be strategic about their implementation, to edit and prepare the videos and then also to prepare materials to accompany them. This might eventually require the time afforded by a Putnam Fellowship, where I could really evaluate a course’s curriculum and work with others to collaboratively plan and create deliberate materials that would fundamentally benefit the course.
However, I am optimistic that as things begin to open up and as people come back to campus, we can regroup and reconsider the future of this project. In particular, I intend to:
1. Make use of one or more of the existing videos in one of my FFS classes in the fall. In FFS 206, for example, we may be able to use part of the FFS lounge video for a lesson on education systems. Although it’s not of optimal quality, testing this video in the classroom will provide valuable feedback for further tests.
2. Collaborate with the French Language Teaching Assistants, who plan to be on campus in fall, to devise another 360 video that we could process and implement that will directly relate to some of the classes I am teaching soon. The TAs work with 206 and 210 classes, and we could specifically target themes and structures that correspond specifically to course material. Since they have been working remotely this year, they already have experience making teaching videos for our classes. This could be an extension of that work.
This further experimentation might allow us to understand to what extent a Putnam might be the logical next step.
III. What have you learned from this experience that would be useful for other faculty members to know?
Some of the background research I have included in part I of this report would be useful for other language faculty especially. I did share preliminary findings related to this grant process at the Talking about Teaching discussion on 2/21/20 and I received positive feedback and helpful suggestions. One of these was to experiment with the 360 cameras for storytelling projects, which I am doing in another course this semester (FFS 295: Voices of Exile and Migration), but which is relying on other technology due to all sorts of pandemic-related constraints. I was also able to share some basic knowledge about the hardware purchased for the grant, and I welcome other faculty to collaborate with me or to experiment with the camera for their own projects. Perhaps once further experimentation is possible in the fall, I can convene language teaching faculty to share some of the background and findings from this grant in order to collaborate on potential projects or ideas moving forward.
IV. What impact would you say this grant has had on your students or your research?
Ultimately the impact of this grant on my teaching is yet to be seen. The Fergusson provided a key opportunity to purchase necessary equipment and to begin to learn how to use it. As we emerge from the pandemic and the intensity of pandemic teaching, this might be a good time to evaluate whether or not this might make a good case for a Putnam fellowship in the future. To really implement this project, more time is needed.
I very much appreciate the opportunity to work with Academic Computing and with students and Teaching Assistants to experiment with this project, and I look forward to more collaboration in the future.
Thank you!
Sincerely,
Anne Brancky
April 16, 2021