DEVINE
In the dynamic landscape of modern industry, the integration of robotic systems into manufacturing processes has become increasingly ubiquitous. This project seeks to redefine the conventional notion of human-robot collaboration by introducing a paradigm shift towards flexibility and adaptability. Unlike traditional robotic setups that adhere strictly to preprogrammed processes, our initiative capitalizes on cutting-edge perception technologies, including cameras and wearable sensors, to enable robots to comprehend and respond to human actions and progress in real-time. This heightened perceptual awareness forms the bedrock for a novel approach to collaborative work, where the robot's interventions are contextually aligned with the human operator's evolving needs.
The central tenet of this endeavor lies in the transformation of robots from rigid tools into intelligent and responsive collaborators. Conventional industrial robots have been constrained by their inability to navigate unexpected deviations, often necessitating human intervention to address unforeseen challenges. In contrast, our framework empowers robots with the capacity to assess and react to unpredicted events. By employing advanced data fusion techniques and machine learning algorithms, the robot becomes adept at identifying patterns, anomalies, and potential disruptions, allowing it to dynamically adapt its actions to match the demands of the task at hand.
At the core of this project is the vision of a harmonious human-robot partnership that thrives on mutual strengths. By leveraging the human operator's creativity and adaptability alongside the robot's precision and speed, we aim to create a synergy that is exceptionally suited to tackling uncertainty. The fusion of human ingenuity and machine precision holds the promise of revolutionizing manufacturing practices and ensuring continuous operations, even in the face of the unexpected.
In summation, this project endeavors to transcend the boundaries of conventional human-robot collaboration by embracing adaptability and responsiveness. By infusing robots with the ability to comprehend human actions, progress, and situational context, we aspire to forge a new era of manufacturing efficiency, where flexibility reigns supreme. The outcome of this initiative holds the potential to unlock innovative avenues for industry, propelling human-robot collaboration to unprecedented levels of productivity and resilience.
Goals
This project aims to redefine human-robot collaboration in flexible manufacturing systems. The focal point is enhancing collaborative robots (cobots) with advanced perception capabilities by integrating data from cameras and wearable sensors. This amalgamation of inputs empowers the Manufacturing Execution System (MES) to intricately gauge assembly progress, enabling informed decision-making.
Furthermore, the MES leverages this real-time progress assessment to tactically assign support tasks to available robots. This dynamic choreography between MES-guided assignments and cobot-driven actions amplifies manufacturing efficacy. The symphony of human intellect and robotic precision unfolds, revolutionizing the adaptive potential and productivity of modern manufacturing.
In essence, this initiative molds a new era of synergy between humans and robots. The cobots, equipped with perceptive faculties, act as collaborators attuned to real-world dynamics. The MES orchestrates this harmony by harnessing assembly insights to distribute tasks adeptly. The result is an ecosystem where adaptability, productivity, and precision converge to shape the future of manufacturing.
Approach
The system is equipped with a range of inputs, including cameras and IMUs, that work together to detect and track the progress of human workers during the assembly process. The current state of the art deep learning models in Human Activity Recognition (HAR) are utilized for our supervised classification challenge. These include variations from simple RNN (Bi-dir LSTM) and CNN to complex attention models. This approach targets the detection of basic assembly micro activities like wrenching and screwing, leveraging data from wrist-worn wearable sensors. Furthermore, diverse models are employed for object detection via stationary cameras. These models detect distinct assembly components, thereby enabling traceability to specific assembly tasks. These different inputs are combined and fed into a rules engine that combines the data and uses a pre-defined precedence graph to determine which task the workers are currently performing. Based on the progress detected, the Manufacturing Execution System (MES) triggers any available robots to perform assistive tasks that have been modeled for the current stage of the assembly process. The robots are equipped with cameras that help them determine the location of parts, where to weld, and which screws to use. The MES is also capable of scheduling robots to provide assistance to multiple human workers, which is detected through cameras and other sensors. As the environment changes and progress is made, the process of robots is dynamically adapted to ensure that they are always working effectively and efficiently.
Expected and Achieved Results
We have designed a controlled lab environment to meticulously test and assess the practicality and efficacy of our proposed methodologies. It's crucial to emphasize the generalizability and scalability of this evaluation setup, which can readily be extended to accommodate real industrial settings for the assembly of more complex products and including more complex robotic responsibilities.
Our designed environment is a dedicated assembly station tailored for the assembly of a compact bicycle, with the collaboration of a robot. In this context, the robot's role entails the precision picking of small assembly components like screws and nuts, subsequently handing them to the worker to facilitate the relevant task.
The assembly's progress is deduced through the combination of micro activity detection (such as screwing, wrenching) and the detection of the picked bike parts. This composite insight is then used by the Manufacturing Execution System (MES), which then allocates the robot the task of picking the screw or washer relevant for the ongoing assembly step. Notably, the robot's skill to identify and grasp the small components is facilitated by its dedicated camera, enabling autonomous recognition and retrieval. These components' positions are not fixed on the assembly station, thereby increasing flexibility. Additionally, we have implemented a failsafe mechanism involving hand gesture commands as a means of communication between the worker and the robot. This contingency approach serves as a vital approach for scenarios like faulty parts (e.g., broken screws).
