recAIcle
Waste contributes significantly to greenhouse gas emissions. As economic growth is the main driver for increased waste generation, a decrease in waste production while maintaining economic growth is a primary objective of the new European Circular Economy Action Plan. Recycling constitutes a key solution to this issue as it reduces the demand for primary, raw resources and mitigates the loss of value in waste management. Accordingly, the EU aims for high recycling rates and has a current target of 70% of packaging waste to be recycled by 2030 and 65% of municipal waste to be recycled by 2035.
Recycling requires materials to be pre-sorted. Despite the already utilized sophisticated sensors and machinery, manual sorting by human employees is still an integral part of waste sorting for recycling (1) to allow the quality assurance of material concentrates and (2) to achieve the high levels of purity required by recycling plants to produce highquality recyclates. In this project, we aim to capitalize on human expertise by integrating human and machine intelligence into recAIcle: a human-guided approach to fine-tuning the sorting process, which additionally provides support to the human worker in the form of augmented guidance.
There is a clear benefit in interfacing and integrating sensor-based classification models with the worker’s task in terms of improved machine learning classification and highlights the importance of the human worker in such applications. In combination with guidance and augmentation technology, increased robustness of the classification models will allow recAIcle to support workers by: (i) guiding them towards potential areas of interest, (ii) providing initial skill adaptation training and support for novice personnel, and (iii) reducing the monotony and strain involved in sorting tasks; leading to increased sorting efficiency and worker satisfaction. The developments planned within recAIcle are guided by at least two complementary use cases. With the methods developed within recAIcle we aim to increase the output of recyclable plastic waste by 25% without increasing the number of sorting workers. This will significantly help to increase the sustainability of waste management and further strengthen Austria as a leader in recycling and as a competitive and innovative technology provider in Europe regarding the sustainable development goals.
Goals
For all use case and application scenarios envisioned in the project, our focus is not on replacing the sorting worker’s labor. While AI will radically alter how work gets done and who does it, the technology’s larger impact will be in complementing and augmenting human capabilities, not replacing them. To support these efforts the recAIcle project aims for the development of a novel continual learning approach, which enables the continuous improvement of automatic classification systems solely by observing a human worker completing the task. In return the system will be used to support an operator carrying out their task. The technologies will be applied to achieve efficient recycling (by significantly improving mixed municipal waste sorting rates) in a circular economy. By reciprocal improvement of human and AI, a robust automatic classification of mixed municipal waste can be achieved, while supporting workers during manual sorting efforts.
Our objective is to achieve a system with wide applicability. For this reason, we will focus on two or three different waste sorting use cases, for instance plastic, metal and/or batteries, to ensure that our methods are applicable to a broad scope of recycling material sorting problems.
Approach
From a purely technical perspective, a human sorting worker constitutes an expert sorting instance, comprised of sensors (i.e., vision), a classifier (i.e., expertise), and actuators (i.e., hands). By choosing an element to be removed from the material flow, the human worker indirectly annotates the element as one requiring classification by an automatic sensor-based sorting approach. The recAIcle project aims to use these and other indirect annotations as the core element for devising a novel, passively human-guided continual learning approach, which enables the continuous improvement of automatic classification systems by observing a human operator working in an application domain.
Expected and Achieved Results
There are several main challenges which have been addressed in the project. First, the recycling material flow varies across the year, depending on the presence of seasonal products as well as new products on the conveyor belt. And second, due to the monotony and strain involved in sorting tasks, the sorting workers execute their task with a certain level of error rate, for instance picking from or leaving the wrong objects on the conveyor belt. To address these challenges, we have develop a federated continual learning framework, which is able to leverage data from the past, while focusing on the composition of the material flow in recent time, from special events in the more distant past, as well as from multiple recycling plants.
Our interactions with the recycling industry and several internal workshops have allowed us to perform a requirements analysis, achieve a design for our first prototype system and define use case and verification scenarios for our prototype system. We have designed a sensors and systems architecture allowing to learn from the sorting workers, see Figure 1, without directly recording them. To this end it features cameras before and after each sorting worker, as well as other sensors installed either in a sorting cabin or at the conveyor belt. Assistance is provided by projectors that can interact with the workers by pointing at certain objects on the conveyor belt. A combination of sensors and cameras allows the system to learn waste sorting by understanding which objects are to be picked up and where they are to be dropped off.
A significant amount of image data has been gathered by the partner Montanuniversität Leoben, at their specialized digital waste research laboratory (DWRL). This data has allowed the consortium to test different approaches towards the deployment of object detectors and classifiers for the recAIcle system, including: object detection methods specialized for detecting objects on conveyor belts, a few-shot learning approach to initialize a classification model with small amounts of data, as well as continual and federated continual learning frameworks for trash particle type classification. Our results and recommendations for the implementation of such frameworks in plastics recycling have been published in conferences and scientific journals, as well as in Recy & DepoTech, the largest waste management and recycling conference in Austria.
Similarly, we have addressed challenges arising from the interactions between our system and the sorting workers. Our system is intended to support the workers without overwhelming them with information. In the context of the project, different sensing technologies for performing action recognition have been considered. In Pro2Future, we have worked in vision-based action recognition, as well as on action-recognition by means of a self-designed augmented glove (Figure 2).
As the methods will be developed in the context of a specific automation scenario, they must be self-contained and resource efficient in order to be successfully deployed on standardized industrial hardware. Resource efficiency will be addressed from the design phase onwards, particularly in the system implementation.
In order to attain the development of methods applicable to multiple use cases, during the project, the consortium has implemented and tested the recAIcle system in collaboration with 2 end users working with different material flows: recycling-plastic waste and a mixed-waste including a main fraction of valuable metal parts.
In the last months of the project, the main task remaining is the experimental evaluation and analysis of the performance of the recAIcle system. That is, the consortium will test and compare the efficiency of a sorting worker with and without the support of the recAIcle system.
