{
  "abstract": "This paper outlines the objectives of the Belgian Defence research project « Tactical Environment Mapping for Battlefield Casualty Care » (DEFRA24 TaMaCare). In collaboration with Ghent University - IMEC i and Pozyxii, the project focuses on integrating a simultaneous localization and mapping1 2 (SLAM) system with multi-spectral imaging (MSI) and Ultra Wideband Positioning3 (UWB) to create a digital twin of the battlefield. The digital twin will provide precise information for first responders about the environment, the position and pathway to reach casualties in need of treatment. Additionally, the system will detect potential threats such as unexploded ordnance (UXO) and landmines. To maintain a feasible project scope, the medical data transmitted by the system will be simulated rather than obtained from actual sensors.SLAM technology enables systems to map unknown environments while determining their location, either through LiDAR4 or optical cameras combined with Inertial Measurement Units (IMU), while functioning independently of external networks (figure 1). This capability is crucial in GPS-denied or jammed environments, as assumed by the project. The initial SLAM setup will utilize a Livox MID-360 LiDAR,5 which offers a large field of view and precise range estimation, essential for efficient environmental mapping. The final setup will transition to a passive sensor system, specifically visual SLAM, using the Core Research Development Kit from Sevensense Robotics.6 This kit includes multiple global shutter cameras and a IMU system7 to enhance localization performance.By significantly increasing the number of bands used in MSI, Hyperspectral Imaging8 (HSI) is a vision technique which captures detailed spectral signatures across the electromagnetic spectrum, making it invaluable for mine detection by distinguishing various materials.9 10 Surface mines can employ camouflage to evade detection by RGB imaging systems, but developing camouflage that effectively covers broad portions of the electromagnetic spectrum is significantly more challenging.11–13 However, the limitations of HSI sensors, such as spatial resolution and cost, make the technology impractical for dynamic and unpredictable scenarios. Therefore, more affordable MSI sensors will be designed with carefully selected bands based on HSI measurement campaigns which will study the spectral signature of threats. Data from these campaigns will be published.In addition to SLAM and MSI, the project will integrate Ultra Wideband (UWB) technology to track casualties. UWB anchors will be laid at adequate locations in the area of interest, and UWB tags will be deployed on detected casualties, allowing for precise indoor and outdoor localization. This real-time system will enhance situational awareness and allow to live-track wounded personnel, improving the intervention of first responders.The TaMaCare system will be designed as a passive, portable, and wearable unit weighing less than 3 kilograms, ensuring ease of deployment in various operational scenarios. The computing unit, powered by an NVIDIA Jetson Orin,14 will handle intensive computations required for sensor fusion and real-time anomaly detection, ensuring energy efficiency and state-of-the-art performance. By leveraging advanced technologies such as visual SLAM, MSI, and UWB, the project aims to significantly enhance situational awareness and safety in military operations.The project will include regular validation experiments in Belgian Defence camps, joining various mass casualty exercises and using a collection of inert landmines. For threat detection, the system is expected to attain an F1-score of at least 80% under both day and night conditions.Abstract A18 Figure 13D Map of the battlefield obtained by SLAM. Casualties are live tracked with UWB tags after being found, and threats are detected with multi-spectral imagingImage Processing and Interpretation Research Group, Ghent, BelgiumPozyx.io : Industrial location tracking solutions, Ghent, BelgiumReferences Frese U, Wagner R, Röfer T. A SLAM overview from a user’s perspective. Künstliche Intelligenz. 2010.Yan C, Qu D, Xu D, Zhao B, Wang Z, Wang D, et al. GS-SLAM: dense visual SLAM with 3D gaussian splatting. In CVPR; 2024.Ranjit G, Kinget P. Ultra wideband: circuits, transceivers and systems. Boston: Springer US. 2008.Yang B, Minh T, Pham T, Yang J. Evaluating and improving the robustness of LiDAR-based localization and mapping. ArXiV. 2024.Livox. Livox Mid-360. [Online]. Available from: HYPERLINK ‘https://www.livoxtech.com/mid-360’ https://www.livoxtech.com/mid-360 .Sevensense. Core Search Manual. [Online]. Available from: HYPERLINK ‘https://github.com/sevensense-robotics/core_research_manual’ https://github.com/sevensense-robotics/core_research_manual .Mostofi N, Elhabib M, El-Sheimy N. Indoor localization and mapping using camera and inertial measurement unit (IMU). In IEEE Symposium on Position Location and Navigation (PLANS); 2014.Manolakis D, Lockwood R, Cooley T. Hyperspectral Imaging Remote Sensing; 2016.Haelterman R. Stand-off landmine detection using (hyperspectral) infrared imaging. In Mine Action Symposium; 2024.Makki I, Younes R, Francis C, Bianchi T, Zucchetti M. A survey of landmine detection using hyperspectral imaging. ISPRS Journal of Photogrammetry and Remote Sensing. 2017; 124.Deans J, Gerhard J, Carter L. Analysis of a thermal imaging method for landmine detection, using infrared heating of the sand surface. Infrared Physics & Technology. 2006; 48.Hupel T, Stütz P. Adopting hyperspectral anomaly detection for near real-time camouflage detection in multispectral imagery. Remote Sensing. 2022.Khodor M, Makki I, Younes R, Bianchi T, Khoder J, Francis C, et al. Landmine detection in hyperspectral images based on pixel intensity. Remote Sensing Applications: Society and Environment. 2021.Nvidia. Jetson AGX Orin. [Online]. Available from: HYPERLINK ‘https://www.nvidia.com/fr-fr/autonomous-machines/embedded-systems/jetson-orin/’ https://www.nvidia.com/fr-fr/autonomous-machines/embedded-systems/jetson-orin/.Disclosures The authors declare that they have no conflict of interest. They do not have any affiliations or involvement with other organizations or entities with an interest in the subject matter or materials discussed in the manuscript.The underlying research for these results received funding from Belgian Defence under contract No. 24DEFRA005. The funding authority, the Royal Higher Institute for Defence (RHID) as well as the project’s partners (Pozyx.io and Ghent University - IMEC) were noticed and approved the publication of this document.",
  "authors": [
    {
      "affiliations": [
        "Dept. of Mathematics, Royal Military Academy, Brussels, Belgium"
      ],
      "name": "Darius Couchard"
    },
    {
      "affiliations": [
        "Dept. of Mathematics, Royal Military Academy, Brussels, Belgium"
      ],
      "name": "Charles Hamesse"
    },
    {
      "affiliations": [
        "Dept. of Mathematics, Royal Military Academy, Brussels, Belgium"
      ],
      "name": "Rob Haelterman"
    }
  ],
  "title": "A18 Enhancing first responder capabilities with digital twin mapping in battlefield environments",
  "uid": "93cdf798-e3b7-5642-bce9-c56b5ad4ba9f"
}