3D Mixed and Augmented Reality (MR/AR) with AI in Archaeological Fieldwork

Abstract

<p>Archaeology is fundamentally a three-dimensional discipline, yet its documentation and analysis have traditionally relied on two-dimensional methods to help view and perceive the distant human past. In recent years, the increasing accessibility of digital tools has enabled archaeologists to scan ancient spaces and artifacts as precise 3D models. These models offer new opportunities for analyzing spatial relationships and object morphology from the human past. Recent advances in Artificial Intelligence (AI) can also help to enhance our ability to use and interpret these data.</p><p>While many researchers have used digital 3D methods for data capture and then interpreted the 3D models in laboratory settings, technological constraints have historically limited their application directly in the field—particularly at excavation sites. However, the emergence of head-mounted mixed and augmented reality (MR/AR) devices presents a promising avenue for integrating digital data directly into excavation environments, especially when using AI-enabled ways to interface with the data.</p><p>In collaboration with student engineers, we have conducted a series of experiments using various MR/AR headsets at our archaeological site in Armenia. These trials have revealed several compelling use-cases where immersive technologies can enhance fieldwork and support real-time decision-making. For example, virtual 3D models can be overlaid onto the excavation trench, allowing archaeologists to visualize and interpret data in-situ. This approach not only enriches spatial understanding but also shifts analytical tasks—typically reserved for post-fieldwork desk analysis—into the active excavation process.</p><p>Moreover, MR/AR glasses offer the potential advantage of hands-free information collection and access, enabling excavators to receive constant updates and contextual data without interrupting their workflow. Despite these benefits, our experiments have also highlighted significant limitations in current MR/AR technology, including issues with outdoor visibility, device ergonomics, and software compatibility, which currently hinder widespread adoption.</p><p>We have turned to two methods to help us overcome the challenges of using AR/MR headsets at the site. First, we built our own custom AR smart glasses and tested them in the field during the summer of 2025. Second, we are implementing Artificial Intelligence (AI) methods throughout our processes to improve the ways that we can interact with 3D data in order to better read the spatial and visual aspects of the human past, in-situ.</p><p>Our new custom AR smart glasses provide a vast improvement on both the wearability of the device outdoors in the sun, and in the amount of information that can be displayed to the archaeologist. For these glasses, we also implemented an AIagent to improve voice recognition to control the user experience. This enabled archaeologists to add or retrieve data using voice commands even within our noisy outdoor environment, or with non-native accented English. With such hands-free data interaction, we eliminate the need to use the small touchpad on our prior off-the-shelf AR glasses—especially difficult when wearing gloves covered in soil.</p><p>We are next exploring how Generative AI (GenAI) can help us to manipulate and create 3D models, for scientific study as well as public education. In a new project, we are experimenting with using GenAI to help students in a university course reconstruct ancient buildings. We hope this can help archaeologists and others to more quickly imagine what ancient sites would have looked like in the past, and how humans would have interacted with these sites.</p><p>Ultimately, our goal is to redeploy digital 3D data in ways that directly support archaeological research and practice during fieldwork. By embedding virtual models into the visual environment of archaeologists, we aim to enhance knowledge, improve decision-making, and foster a more dynamic and informed research process.</p>