BRAINTEASER Open Evaluation Challenges launch

The Intelligent Disease Progression Prediction (iDPP) challenge aims at designing and developing an evaluation infrastructure for AI algorithms able to:

●      better describe disease mechanisms;

●      stratify patients according to their phenotype assessed all over the disease evolution;

●      predict disease progression in a probabilistic, time dependent fashion.

By “evaluation infrastructure” we mean experimental collections, evaluation protocols, evaluation measures, ground-truth creation protocols to develop shared approaches, promote the use of common benchmarks, foster the comparability and replicability of the experiments

iDPP will be organized under the umbrella of CLEF (Conference and Labs of the Evaluation Forum), the European initiative for the experimental evaluation of multilingual and multimodal AI algorithms.

iDPP features the following tasks:

●      Pilot Task 1 – Ranking Risk of Impairment: it focuses on ranking of patients based on the risk of impairment in specific domains. More in detail, we will use the ALSFRS-R scale to monitor speech, swallowing, handwriting, dressing/hygiene, walking and respiratory ability in time and it asks participants to rank patients based on time to event risk of experiencing impairment in each specific domain.

●      Pilot Task 2 – Predicting Time of Impairment: it refines Task 1 asking participants to predict when specific impairments will occur (i.e. in the correct time-window). In this regard, we assess model calibration in terms of the ability of the proposed algorithms to estimate a probability of an event close to the true probability within a specified time-window.

●      Position Paper Task 3 – Explainability of AI algorithms: We evaluate proposals of different visualization frameworks able to show the multivariate nature of the data and the model predictions in an explainable, possibly interactive, way.

The final iDPP event will be co-located with CLEF 2022 which will be held in Bologna, Italy, from 5 to 8 September 2022.

Registrations for participating in iDPP@CLEF 2022 are open.