Threatened frog species, citizen science, and cultural consensus theory.

Field recordings of threatened species can produce large quantities of data which can make coding the recordings time-consuming. In cases where machine algorithms are unavailable, citizen scientists are recruited via crowd-sourcing. However, because citizen scientists lack comprehensive training the results of their coding may be difficult to interpret. Competence may vary between citizen scientists, but without knowing the ground truth, it is difficult to identify which scientists are most competent. We used cultural consensus theory to analyze data from a crowdsourced analysis of audio recordings of Australian frogs. Hundreds of citizen scientists were asked whether the calls of nine frog species were present in brief audio recordings. Through modelling, characteristics of both the scientist cohort and recordings were estimated. We then compared the model’s output to expert coding of the recordings and found agreement between the cohort's consensus and the expert evaluation. This finding adds to the evidence that crowdsourced analyses can be utilised to understand large-scale datasets, even when the ground truth of the dataset is unknown. The model-based analysis is promising as a tool to screen large data sets prior to investing experts’ time, and also as a way to more efficiently allocate resources when recruiting citizen scientists or training classification algorithms.