APECGR
This project researched user experience design based on contextual triggers in the welding domain. Usually in industries, good user experience design is not a significant focus. Don Norman argued that, while human-centered (interface) design processes in theory represent the ideal, in practice they are commonly untenable. He goes on to iterate that in his experience, market-driven pressures plus an engineering-driven company yield ever-increasing features, complexity, and confusion. But even companies that do intend to search for human needs are thwarted by the severe challenges of the product development process, in particular, the challenges of insufficient time and insufficient money. Norman anecdotally postulates that there are only two actual drivers of new product designs: (i) adding features to match the competition, and (ii) adding some features driven by a new technology. To break this mold, APECGR puts the user at the center of the design process, investigating ways to leverage current AI techniques to optimize user interaction and interface design, empowering workers and designers to create, and efficiently utilize user-centric interfaces. The developed systems should make the user want to interact with their product and caring for it. To gauge this emotional state of welders, the project demonstrated a sensor-based approach to determine a welder's emotions during a dry lab study setup. This part of the project culminated in the development of a model-based reinforcement learning approach for dynamic interface adaptations, that can be used by interface designers as input to their development processes.
Additionally, APECGR investigated IMU-based methods for movement tracking of manual welders to determine the current weld seam and skill level. A demonstration of a skill level detection algorithm, and the concept of an assistance system for novice manual welders was created. Such skill level systems can provide tailored interactions based on actual need and profile, thereby avoiding annoying and negative interaction experiences. The project tested the developed system in the domain of manual arc welding using both novices and experts at welding. The results were demonstrated via multiple independent studies of the individual parts of the overall system in industrial environments and encompass the development of a final prototype.
Goals
The goal of this project was to investigate ways to detect manual welding movements, and leverage them to determine the current production state, as well as to determine the emotional state of welders. The project aimed for a user-centric analysis of industrial welding interfaces, towards optimization of these, using e.g. reinforcement learning as technology. The developed systems should be demonstrated in realistic application environments, via the development of a prototypical assistance system for welders. The system should detect the skill level of a welder, as well as the current production step, in order to only provide feedback to novice welders if required. Part of this system was the development of a cognitive state recognition engine, that measures the users emotional affect in addition to the skill level. The overarching goal hereby, was the investigation of how well these individual systems can be implemented purely based on data provided by machine-internal sensors.
Approach
The project implemented the given goals using a sensor-data driven approach. A purpose-built data collection system was created, consisting of (i) the welding torch itself as a data source, (ii) an eye tracker as indicator for affect, intent and interface quality, (iii) body-physiological wrist-worn sensors, (iv) as well as in-ear microphones. In this way, the project was able to identify indicators for affect and skill in the welding domain, based on established research, and be able to investigate the impact of user experience design choices on the emotional state of welders. The collected data was used with a model-based reinforcement approach to generate dynamic user interface candidates to match a given industrial context and workflow requirements.
Expected and Achieved Results
The project yielded multiple systems that interact in order to provide user focused feedback regarding the mindful handling of a manual welding torch. Specifically, the project resulted in (i) a system that recognizes movements of the welding torch that may be harmful and to attempt tracking of welders movements to determine the currently created and worked on weld seams, (ii) a system to determine the skill level of a welder based on their welding movements, (iii) a study on sensors and derived indicators for affect that are suitable for the welding environment, (iv) an evaluation of the two iterations of the graphical interface of the welding device with the purpose of enhancing user experience, and (v) methods of automatically generating useful UI designs based on the current welding context as well as global requirements. This system will optimize itself for known, re-occurring interactions during welding, while keeping in mind often used interaction elements, to make sure they remain easily available.
