Aim of the project
Recent advances in image classification have fostered the advent of user-friendly mobile apps helping amateurs to identify species from images with great potential for citizen science. These apps are working well for common species with distinct morphological features. However, they tend to perform badly for less common species and for groups of species with very similar morphological characteristics. Many Swiss plant species, for example, are not recognized well by the current apps. Here, we propose a novel approach combining machine learning-based image recognition with spatially explicit ecological and morphological meta-information for the identification of the c. 4'000 Swiss plant species from georeferenced pictures. This combination is expected to considerably improve species identification as new images are not only classified according to visual features but also with regards to ecological and geographical plausibility. This will greatly aid data acquisition by citizen scientists.
We have just trained the first set of neural nets and already achieve a decent classification accuracy for 1659 species. Based on image information only, the best net so far identifies 72.2% of test images correctly, and for 90.4% of test images the correct species is among the five species, that are estimated as the most probable ones by the net. When including locality information as well, the statistics increase to 76.0% for correct classifications, and 92.4% matches within the five most probable suggestions. Even though these numbers are encouraging, they also show that for many species classifications are not yet functioning well. Moreover, the image material is currently insufficient for any assessment in the case of almost 2000 species. A detailed overview over classification accuracies on the species level is provided in this document. Further information on the statistics used and priority species for photographing can be found under the 'Priority'-Tab in the section 'Supporting the project'.
It is important to note that these are preliminary results that have to be taken with a grain of salt. In the coming months we are going to clean and adapt training and in partiuclar test data thoroughly and therefore the quality scores may be changing distinctly, in particular on the species level. The information on whether or not sufficient suitable images are available for the different species is a bit more reliable.
2020 - 2022