The overarching aim of ENGAGE is to develop new methods, processes, and algorithm, specifically using robotisation and AI, and to provide proof-of-concept for representative cases to increase sustainability, productivity, and safety of MWM industries. ENGAGE is divided into 4 specific research objectives:
1) To estimate and track the state of the machine and its surrounding dynamic environment in real-time using multimodal sensing system, enabling autonomous navigation in worksites (forest, construction, terminal) and updating as-built construction information models.
- we will develop algorithms to enable real-time perception capabilities to determine the precise location of the machine as well as the 3D geometry and semantics (i.e., the meaning of data, such as object types) in the worksite
- we will create methods to continuously update the existing 3D representation of the worksite as it changes over time (e.g., as objects move or the landscape is modified)
- we will develop techniques to represent and group the contents of the 3D models in ways that support downstream tasks, such as automated planning, situational awareness, and autonomous operations.
2) To enhance the adaptability and precision of machine learning-based control algorithms for intelligent mobile machinery in particular excavators and excavation tasks, aiming to reduce deployment costs and expand application versatility in complex dynamic environments.
This objective involves the development of novel learning algorithms that can
- leverage large, low-quality yet inexpensive offline datasets to train high-performing controllers, decreasing cost for data-collection for at least 50%
- enable human operators to intuitively fine-tune controllers with minimal input, aiming to improve success-rates to 95%, and
- facilitate rapid policy transfer to new tasks and robotic configurations, thereby minimizing retraining requirements and increasing operational robustness.
This approach is intended to set a new benchmark in data-efficient control, mastering complex control challenges using actively curated, limited datasets and inexpensive offline knowledge.
3) To develop methods for mitigating unsafe motions, and ways of certifying safety of autonomous functionalities of MWMs and bringing the new functionalities offered by AI into practice in construction, and agriculture.
We will push the state of the art in how AI-based autonomous functionalities can be employed, and validated safely, and safety certified.
- we will develop certifiable safety controllers for learned traversability maps in a construction site, scalable to high dimensional dynamic models of wheeled machines moving on uneven terrain
- we will design AI-integrated control systems adhering to safety standards for autonomous agricultural vehicles and contribute to future machinery standards
4) To develop simulation-based solutions for automated testing, training, and optimization of autonomous capabilities in construction and forestry applications.
Our goal is to enable scalable use of simulators of MWM that realistically capture the interaction with the environment (contact forces and soil deformations). We will develop methods for
- automated generation of simulation scenarios, state estimation and parameter identification from real-world data
- pre-training of AI models for predicting detailed contact response and domain adaptation to real applications; and 3) extracting actionable information from the simulator.
Together, the methods support both learning from and directly acting on the created synthetic data.