SAFE-TRACK
Drones are excellent machines to carry cameras, laser scanners and sensors to provide data for industrial and commercial applications. Thanks to GPS technology their operation in outdoor applications has been increasing steadily in recent times. However, the adoption of drone technologies in environments where GPS positioning is not reliable has been slower and always subject to the requirement of having a drone pilot deploy the drone for the application. This issue is even more present in indoor working areas, such as warehouses and other industrial facilities.
Recent technological innovations in drones, computer vision and machine learning are cutting down the gap to make indoor drone automatic industrial operations economically feasible. On-board odometry allows a drone to have a rough estimate of its local positioning, which estimation is based entirely on data from on-board sensors, and which is calculated entirely on computers on-board the drone. A drone may therefore gather data relevant to an industrial application relying only on its on-board odometry. However, the odometry calculation may malfunction or it may in some cases provide bad estimates, which impacts the value of the gathered data – an example of this are pictures whose location data is inaccurate.
A successful implementation of drones performing tasks inside industrial facilities and warehouses, especially when working in the potential presence of human workers, requires additional layers of security. In this context, it is necessary to minimize the risks of an accident involving the drone. A possible approach, to achieve safe indoor drone automatic industrial operations is to add a second sensing layer off-board the drone, for instance with a monitoring station. Therefore, there is an interest in studying external means to achieve drone localization. In the case of a malfunction of the on-board drone odometry, this event can be detected by comparison to the localization data. Moreover, the overall drone system (consisting of the drone and the monitoring station) can be studied under the point of view of fail-safety.
Goals
In this context, the project SAFE-TRACK has had a focus on achieving automated drone-based inventory management (left Figure), without maintaining line of sight to the drone. This is required to enable flights within operating hours – which means that the drone is collaborating with humans and logistics equipment. Over a period of seven months, Area 4.1 (Cognitive Products, P2F), TU-Graz (Institute of Computer Graphics and Vision), D-ARIA and Roto Frank Austria have developed and tested methods to secure and monitor drone operations in warehouse environments.
Approach
We have investigated the capability of tracking a drone during operation by means of external tracking cameras – to support the operation of the drone and towards achieving a higher degree of autonomy in its operation. Part of our focus has been on fail-safety, that is in short, on achieving a drone system (consisting of the drone and the monitoring station) with no single point of failure, which could cause an accident involving the drone. Our second focus has been on working towards achieving automated operation well aligned with safety regulations, but which does not require a safety pilot (or an operator) maintaining line of sight to the drone during operation.
Expected and Achieved Results
Our experiments have shown the capabilities of our system to track the drone with high reliability and positioning accuracy. In our system, the drone is tagged to ease the vision-based re-detection and tracking task and the tracking camera is placed statically inside the working environment. In the right figure, we show our results on the task of tracking a drone by means of external cameras. The cameras have been registered, that is localized between each other and to the environment, in order to be able to embed our 3D data in the images (and videos) of the experiment. In the images, the following reference frames are depicted (see right Figure): (blue) frame of the drone, (red) reference frame fixed to a relevant point on the ground and (green) reference from of the tracking camera.
Our work represents a first step towards achieving the safe operation of drones in warehouse industrial environments. The experiments demonstrate the feasibility of utilizing automated external monitoring in indoor drone operations. Among other advantages, our system provides more information to the operator, such as flight statistics over time and the repetitive acquisition of metrics that reflect the performance of the drone in each particular area of the warehouse where the system is deployed.
For drone operations as a whole, our external monitoring solution signifies an additional layer of security, which can be used as part of the navigation architecture towards achieving fail-safety in drone industrial operations.
