RT Journal
A1 Seppey, Mathieu
A1 Manni, Mosè
A1 Zdobnov, Evgeny M.
T1 LEMMI: a continuous benchmarking platform for metagenomics classifiers
JF Genome Research 
JO Genome Research 
YR 2020 
FD August 01 
VO 30 
IS 8 
SP 1208 
OP 1216 
DO 10.1101/gr.260398.119 
UL http://genome.cshlp.org/content/30/8/1208.abstract 
AB Studies of microbiomes are booming, along with the diversity of computational approaches to make sense out of the sequencing data and the volumes of accumulated microbial genotypes. A swift evaluation of newly published methods and their improvements against established tools is necessary to reduce the time between the methods' release and their adoption in microbiome analyses. The LEMMI platform offers a novel approach for benchmarking software dedicated to metagenome composition assessments based on read classification. It enables the integration of newly published methods in an independent and centralized benchmark designed to be continuously open to new submissions. This allows developers to be proactive regarding comparative evaluations and guarantees that any promising methods can be assessed side by side with established tools quickly after their release. Moreover, LEMMI enforces an effective distribution through software containers to ensure long-term availability of all methods. Here, we detail the LEMMI workflow and discuss the performances of some previously unevaluated tools. We see this platform eventually as a community-driven effort in which method developers can showcase novel approaches and get unbiased benchmarks for publications, and users can make informed choices and obtain standardized and easy-to-use tools.