Mixed Reality for Enhanced Lab Courses

Advancing understanding of learning

The Problem: Practical courses are a common feature of many curricula and aim to give students hands-on experience in applying what they have learnt. Generally, students follow written manuals in the laboratory setting, observe the process carried out once by the instructor, and then perform the process on their own. A disadvantage of this approach is that often students passively follow instructions without developing the in-depth understanding and critical thinking skills required. Practical courses are essential, but effective and engaging practical instruction for large numbers of students remains challenging.

Research Goals / Questions: We aim to advance learning outcomes in practical bioengineering courses as part of the medical curriculum by using Mixed Reality (MR) tools. Mixed reality combines the real world (using translucent displays in glasses) with anchored virtual objects (in real 3D), allowing the user to treat them as “real” objects. These tools enable three teaching strategies supported by learning sciences:

  1. Continuous learning assessment and engagement through gamification
  2. Multi-modal representations to enhance understanding
  3. Control of variables in virtual reality to improve the ability to implement and develop new processes

Implementation Strategy: The existing course on “Practical Methods in Biofabrication” is an exciting module for students of the medical curriculum as it bridges medicine and engineering, providing an excellent test case for implementation. Students gain tangible exposure to emerging medical technologies by working with materials, cells, and biological building blocks to assemble grafts for tissues and in vitro models. This project will replace written manuals with MR tools (such as the Microsoft HoloLens) that guide students through fabrication processes, coupled with principles of gamification to promote student engagement. For example, after completing critical steps in a practical procedure, students will be asked a question probing their understanding of the physical mechanism behind that particular step in the process. Only after successfully answering these questions will the student be allowed to advance, similar to unlocking a new level in a video game. In the case of an incorrect answer, they will get feedback from the HoloLens which lead them to a better understanding.

Additionally, MR learning modules will be developed that leverage multi-modal representations in the form of interactive media virtually overlaid onto laboratory components aimed to improve an enhanced understanding of the processes applied to them. Students will have learnt about the theory behind the process in classroom units beforehand and will then be confronted with another visualization of the same concept in the context of the laboratory environment. Visualizing and representing a concept in multiple ways has been described in learning sciences as an effective tool to enhance learning.

Lastly, MR will allow students to interactively vary certain experimental parameters while observing the virtual outcome, cultivating an understanding of the rationale motivating key steps in a process protocol. This feature helps students build understanding of dependencies and overcome retrieval failure without wasting the time and materials required to repeat the procedure physically.

Translating research into practice

The effective implementation of methods from learning sciences using MR tools has the potential to significantly improve learning outcomes and more effectively equip students to design practical processes. Synergy with the Embodied Learning project as well as with the professorship on learning and technology will create an interdisciplinary, collaborative learning environment. Following this pilot, these methods could be extended to other practical courses at ETH, improving learning outcomes and scalability of practical instruction. More than simply providing a novel interface, these tools would be used to assess comprehension, build strong associations between theory and experience in the laboratory, and enable a large number of students to gain a deeper understanding of the protocol they are following.

Andrea De Micheli

Simone Schürle

Thomas Korner

Fabio Grillo

Manu Kapur