AssE
The purpose of "Opportunistic Awareness in Assistive Environments" was to support scientific work on the following topics:
- Feedback and Guidance Systems Industrial ICT systems that are designed to provide feedback or guidance to humans have been shown to fail if the provided assistance information is not suitable to the worker and situation in question. AssE aims to further the state of the art in this aspect via provisioning of skill sensitive feedback, i.e. feedback that is designed for- and adapted to the worker’s skill and experience. Part of this approach is to guarantee that provided guidance information to industrial workers is explainable and reconstructable.
- Embedded AI and Cognitive Products Embedded AI is expected to be a major future driver in industries. Waiving the need for large scale data centers and achieving decentralized artificial intelligence would open new applications for AI, removing restrictions imposed by the ongoing “data silo-fication” in many domains. AssE aims to further the state of the art by proposing a research endeavor in this field. This effort will be co-organized with research proposals on advancing neuromorphic architectures of cognitive products – a field that is still very young and in need of more research.
- Workflow Recognition An important puzzle piece for many industrial ICT applications involving human workers is the correct detection of their current work tasks. Applications are for example the construction of quality assurance logs, or the detection of human oversights and manufacturing errors. AssE will focus on advancing the state of the art in micro- and macro work step detection, as well as activity recognition using overhead cameras.
- Future AI AssE aimed at identifying long term- and short-term future application domains for AI systems and provided text blocks and state of the art reviews for research grants proposals on both time scales: For the long term, human-like intelligence informed AI systems and neuro-robotics are investigated, whereas on the short term, cognitive systems for green technologies are researched.
Goals
The goal of the project was to further strategic developments in terms of scientific output. The following areas of research were investigated:
- Sensor-based recognition of the state of (i) machines (system state), (ii) humans (skill level, cognitive state, vital state), and (iii) processes (workflow recognition).
- Display of- and interaction with digital content, such as (i) video snippets for documentation purposes, and (ii) haptic signals for unobtrusive notifications.
- Aspects of AI and its applications, such as (i) human-informed AI, (ii) green tech and sustainability, (iii) neurorobotics.
The achievement of goals was demonstrated through (i) scientific publications, (ii) demonstrators and videos, (iii) reviews and summaries of the state of the art, and (iv) text blocks for research grant proposals.
Approach
To achieve the target scientific output, the project was built on a strong scientific consortium, involving partners from the JKU and TUM, where Pro²Future focused on research related to (i) feedback and guidance systems, (ii) cognitive products, and (iii) workflow recognition, JKU focused on (i) embedded AI, (ii) human state recognition, and (iii) green-tech AI, and TUM focused on (i) robotics, (ii) neuromorphic architectures of cognitive products, and human-like intelligence informed AI.
Expected and Achieved Results
The project resulted in 1 journal paper and 4 conference papers that were submitted to the MDPI-Applied Science journal, as well as the PETRA and IOT conferences.
Micro Activities Recognition in Uncontrolled Environments (A. Abbas, M Haslgrübler, AM Dogar, A Ferscha; Applied Sciences 11 (21))
Determining Best Hardware, Software and Data Structures for Worker Guidance during a Complex Assembly Task (B Anzengruber-Tanase, G Sopidis, M Haslgrübler, A Ferscha; Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments)
Micro-activity recognition in industrial assembly process with IMU data and deep learning (Georgios Sopidis, Michael Haslgrübler, Behrooze Azadi, Bernhard Anzengruber-Tánase, Abdelrahman Ahmad, Alois Ferscha, Martin Baresch; Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments)
Skill Level Detection in Arc Welding towards an Assistance System for Workers (M Laube, M Haslgrübler, B Azadi, B Anzengruber-Tánase, A Ferscha; Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments)
Privacy Preserving Workflow Detection for Manufacturing Using Neural Networks based Object Detection (A. Ahmad, M Haslgrübler, G Sopidis, B Azadi, A Ferscha; Proceedings of the 11th International Conference on the Internet of Things)
