Development and evaluation of weed application maps for the use of robots for mechanical weed control (EWIS2).

Already in the research project Evaluation and further development of modern artificial intelligence methods for the automatic detection of weeds in sorghum with the aid of drones (EWIS), it was possible to develop accurate models for the automated detection of weeds and sorghum in drone images with the aid of modern artificial intelligence methods. This project has provided an important building block for a variety of other research questions and applications.

The data collected and annotated in the EWIS project, as well as the AI models developed, open up the possibility of developing novel Kl-based open-source software for automated generation of small-scale weed maps. Based on these weed maps, application maps for site-specific weed control in maize and sorghum by modern field robots will be generated in the EWIS2 project. Cooperation between the Straubing and Ruhstorf sites in the field of digitization is being driven forward and synergies are being exploited. Through intensive knowledge transfer, an interdisciplinary exchange is achieved.

The following work packages (WP) are being worked on in the joint project:

AP 1: Optimization of data collection with drones for the efficient generation of high-resolution stand images and their annotation as well as coordination of the project network (TFZ)

For the successful implementation of the project goals, a reliable database is elementary. Therefore, drone flight as a basis for site-specific weed control is to be adapted in such a way that stand records of optimal quality are collected as automatically as possible. The focus is on increasing efficiency by improving image quality while at the same time increasing area performance. This is an important step for the scaling of the technology. Further development of smart pre-masking will make annotation and classification of images more efficient and accurate. Annotations of the information in an image are necessary for training and evaluation of AI methods in WP 2. The use of a drone with RTK module for accurate GPS data will be used to collect as-built imagery with accurate georeferencing. The impact of this technology on the weed maps generated in WP 2 will be investigated. For this purpose, existing data from EWIS can be used as a reference and the marginal utility of this technology compared to conventional photo drones will be examined. The project network is coordinated by the TFZ.

AP 2: Development and validation of an AI for automated generation of small-scale weed application maps (HSWT)

For the generation of the weed maps, the first step is to decompose the image captured by the drone into smaller image sections in order to efficiently process the images using artificial neural networks. After this preprocessing step, the AI models are able to create segmentation masks for these image sections that indicate the position of each plant as well as its class (weed or sorghum/corn) (Genze et al. 2022, Under Review). In this AP, we will investigate what kind of annotation is needed to create application maps, especially to facilitate the adaptation of the AI to other crop types. Furthermore, the AI will be adapted and extended to produce accurate GPS-based predictions. This is necessary to be able to generate weed application maps in a further step. For this purpose, among others, the additional imagery obtained in WP 1 will be used. A prototypical responsive browser-based application will be developed and tested in order to visualize the weed map and make it available to the user as intuitively as possible.

AP 3: Integration of the application maps into field robotics (TFZ, HSWT, LfL)

For the practical implementation of the application maps created in WP 2, a successful integration into existing robotics platforms is crucial. According to the mode of operation, an optimal procedure will be developed to deploy the robot highly efficiently in the field. Improved route planning can be used to navigate to and regulate weed hotspots to minimize time-consuming empty runs by the robot. The potential of optimized lane planning will also be evaluated. The field robotics available to date have various, mostly limited, options for lane planning. First, the interface problems relevant for realization in market-available hackrobotics and safety aspects in autonomous operation must be pointed out and addressed.

AP 4: Environmental-economic system evaluation (LfL)

For an evaluation of the overall system with regard to environmental impact and economic efficiency, the method is compared in field trials with established methods of weed control. The economic evaluation is based on the generated weed mapping and is calculated for the two possible application areas mechanical or chemical-synthetic weed control. In order to reliably represent the bandwidths of the weed distribution on agricultural land, the economic evaluation is based on the weed mapping of several fields. Only site-specific mechanical weed control reduces the risk of erosion on slopes and reduces the trade-off of the method. For the environmental-economic evaluation, the potential of erosion prevention by site-specific mechanical weed control is quantified on the basis of an erosion estimation model.

AP 5: Knowledge transfer and public relations (TFZ, HSWT, LfL)

Knowledge transfer will be carried out throughout the project duration. Publications will be made primarily in technical journals and international journals. In order to bring the findings from the project closer to the agricultural target group, it is planned to demonstrate the use of drones and chopping robots live in the field as part of the TFZ's annual trial tours. This offers the opportunity to enter into direct exchange with farmers.