Grounding Abstract Mathematics through Embodied Interaction in Virtual Reality


Embodied interaction, embodied cognition, abstract mathematics, virtual reality, multiple representations, concreteness fading, higher education


Learning abstract mathematics is difficult. This issue might have several origins. First, because of their abstract nature, the topics can be difficult to grasp: if one cannot see and cannot manipulate, one might be restricted in their ways of building and grounding knowledge. We refer to this problem as the grounding problem, that is the question of how abstract mathematical formalisms can gain meaning. Second, the pedagogy used to teach these topics also has an impact on the learning outcome: is the teacher primarily providing explicit instructions to the students? Or do the students have space to explore on their own? We refer to this problem as the pedagogy problem. Third, the state of the learner, and in particular their beliefs, can impact their motivation, their confidence, and therefore the outcome of the learning activity. In particular, students hold several unproductive beliefs about mathematics [Schoenfeld 1992] that could have a role in this issue, namely a conception of mathematics as a solitary activity that has little to do with other disciplines or the real world.

Often, the way mathematics taught in schools is in conflict with our knowledge of how people learn, and even the purpose of the discipline in general [Trninic 2018]. Students are taught mathematical concepts through practices often emphasizing mathematical formalisms rather than mathematical thinking, and then asked to solve related problems, mostly on their own. This discrepancy is not a sign of the ignorance of our educators, but rather the result of a lack of resources to implement alternative strategies. We argue that technology could help here, by enabling students to manipulate and visualize otherwise purely abstract mathematical concepts (grounding problem), as well as providing pre-instruction opportunities for sense making (pedagogy problem).

In this project, we use a pedagogical design of problem solving followed by instruction [Sinha 2021]. Addressing the grounding problem, we focus on the problem solving phase and investigate ways of grounding abstract mathematical concepts through embodied interaction. To do so, we use Virtual Reality to design embodied activities around major and influential mathematical topics. Through these deep dive studies, we aim to grow a better understanding of the link between embodied interaction and learning, especially when targeting more abstract topics. We then intend to extract design recommendations for embodied activities in the context of mathematics education.

The picture shows two cells, one labeled "Learning Sciences" and one labeled "Computer Science". The cells merge into one other cell labeled "Interdisciplinary". The Learning Sciences cell contains a text: "Embodied Cognition, Through which mechanisms can embodiment support learning of abstract mathematics topics?". The Computer Science cell contains "Embodied Interaction, How does interaction design in virtual reality relate to the sense of embodiment?". The interdisciplinary cell contains a question "How to design embodied interaction to support learning of abstract mathematics concepts in virtual reality?" and a list of three topics: analysis, graph theory, and linear algebra.

Project structure

Research questions

Our work undertakes a connecting role between Learning Sciences and Computer Science. Firstly, this project builds on our knowledge of how people learn to design and implement educational tools for abstract mathematics. In return, we use our novel technological capabilities to consolidate and test theories of Learning Sciences by enabling specific mechanisms of learning.

Specifically, we wish to build an automatic tool to generate embodied activities to support grounding of mathematical objects and concepts. The rationale behind this project is to offer a pre-instruction space for sense-making, allowing students to explore mathematics domains without having to rely solely on formal and symbolic representations. Importantly, the goal of this project is not to argue against the use of formal and symbolical mathematical language: this is an essential tool to communicate and build mathematics efficiently. Here, we solely aim at easing the way into such formalism by providing the learners with opportunities to build intuitions and cognitive metaphors from which abstraction can emerge and in which abstraction can be grounded. This approach follows Thompson’s recommendation to provide multiple representations for students to draw connections and isolate a common pattern [Thompson 1994].

Overall, the goal of our work is to address the following questions: How can we ground abstract mathematics? How can embodied interaction help in this endeavor? How can we design the representations and the interaction with them to improve grounding? How do these results vary across different domains of mathematics?

State of the project

As a first step, we focused on embodied interaction in Virtual Reality. We designed and investigated the usability of “Digital Gloves”, an interaction mechanism where the digital world is projected directly onto the body of the user [Chatain 2020]. Through this study, we found that the expectations that the users had from the physical world transferred to the virtual world, making certain interaction techniques more difficult to use. Moreover, we noticed that after only a few minutes of playing, participants recalled and used the gestures from the games during the follow-up interview. 

The images is split into 4 vignettes. The first one shows a person with a VR headset, looking at her hand. The second one shows a pinball game displayed on one hand. The third one shows a maze game on one hand. And the fourth one shows a two hands game where each hand contains an item and the hands are positioned to mimic a twin-pan balance.

Digital Gloves [Chatain 2020]

We then looked into embodied interaction for learning. We started focusing on derivatives and implemented two prototypes for this project. As part of a mixed-methods approach, we conducted an explorative qualitative study to gather feedback from experts. We then performed a follow up quantitative study where we compared two different degrees of embodiment [Johnson-Glenberg 2017], one stronger, in Virtual Reality, and one weaker, on a tablet, as well as different types of embodiment [Melcer 2016, Ottmar 2019]: direct-embodied, and enacted. We evaluated usability, sense of embodiment, sense of agency, and learning outcomes [Chatain 2022]. We are now extending this work with another quantitative study focused especially on learning.

Three different ways of learning derivatives using embodied interaction: direct-embodied interaction on tablet, direct-embodied interaction in Virtual Reality, and enacted interaction in Virtual Reality [Chatain 2022]

We then focused on the context of the interaction and conceptualized embodiment as a form of concreteness. We defined embodied concreteness as “a form of concreteness that involves a high degree of embodiment, in a situated and relatable context.” Through a user study with 89 bachelor students, we showed that embodied concreteness presents unique affordances for grounding abstract mathematics, specifically graph theory [Chatain 2023].

The figures contains three pictures, in doodle style. On the first figure, a character solves a graph theory problem on paper with an abstract representation. On the second figure, on tablet with an abstract representation. On the third figure, in Virtual Reality with a relatable and embodied representation. The graph is always the same and the character always interacts with the same edge.

Three approaches to graph theory exercises: abstraction on paper, manipulated concreteness on tablet, and embodied concreteness in Virtual Reality [Chatain 2023]

Finally, in order the reconnect the embodied activities with formal representations of mathematical objects, we designed and evaluated two prototypes for input of mathematical expressions in Virtual Reality [Sansonetti 2021].

The image has three vignettes. First, the VR environment. A room with windows, green walls, wooden floor, and a plant in the corner. The second vignette shows a stick-figure user pressing buttons on a virtual keyboard to add mathematical symbols to the expression in front of them. The last vignette shows the user dragging and dropping the mathematical symbols directly onto the expression.

Two input techniques for mathematics in Virtual Reality [Sansonetti 2021]


Chatain, J., Sisserman, D. M., Reichardt, L., Fayolle, V., Kapur, M., Sumner, R. W., … & Bermano, A. H. (2020, November). DigiGlo: Exploring the Palm as an Input and Display Mechanism through Digital Gloves. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI Play) (pp. 374-385). or (pdf) or (video)

Sansonetti, L., Chatain, J., Caldeira, P., Fayolle, V., Kapur, M., Sumner, R. W. (2021, September). Mathematics Input for Educational Applications in Virtual Reality. In International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE).  or (pdf)

Chatain, J., Ramp, V., Gashaj, V., Fayolle, V., Kapur, M., Sumner, R. W., Magnenat, S. (2022, June). Grasping Derivatives: Teaching Mathematics through Embodied Interactions using Tablets and Virtual Reality. In Interaction Design and Children (IDC’22). or (pdf) or (video)

Chatain, J., Ramp, V., Gashaj, V., Fayolle, V., Kapur, M., Sumner, R. W., Magnenat, S. (2022, May). ETH-EPFL Grasping Derivatives with Embodied Interactions. Joint Doctoral Program for Learning Sciences Kick-off Event (JDPLS). (pdf)

Chatain, J., Varga, R., Fayolle, V., Kapur, M., Sumner R. W. “Grounding Graph Theory in Embodied Concreteness with Virtual Reality”. In TEI ’23: Proceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction. (2023). or (pdf) or (video)


Johnson-Glenberg, M. C., & Megowan-Romanowicz, C. (2017). Embodied science and mixed reality: How gesture and motion capture affect physics education. Cognitive research: principles and implications, 2(1), 1-28.

Melcer, E. F., & Isbister, K. (2016, May). Bridging the physical divide: a design framework for embodied learning games and simulations. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems(pp. 2225-2233).

Ottmar, E. R., Walkington, C., Abrahamson, D., Nathan, M. J., Harrison, A., & Smith, C. (2019). Embodied Mathematical Imagination and Cognition (EMIC) Working Group. North American Chapter of the International Group for the Psychology of Mathematics Education.

Schoenfeld, A. H. (2016). Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics (Reprint). Journal of Education196(2), 1-38.

Sinha, T., & Kapur, M. (2021). When Problem Solving Followed by Instruction Works: Evidence for Productive Failure. Review of Educational Research, 00346543211019105.

Trninic, D., Wagner, R., & Kapur, M. (2018). Rethinking failure in mathematics education: A historical appeal. Thinking Skills and Creativity30, 76-89.

Thompson, P. W. (1994). Students, functions, and the undergraduate curriculum. Research in collegiate mathematics education1, 21-44.

Julia Chatain

Prof. Dr. Manu Kapur

Prof. Dr. Robert W. Sumner