When a new face recognition algorithm claims state-of-the-art performance, or a morphing attack detector reports a near-zero error rate, the natural question is: state-of-the-art on what? Error rate on which dataset, evaluated under which conditions, by whom? These questions matter enormously. The performance of a biometric AI system is inseparable from the data it was trained on and the benchmark it was evaluated against.
Why biometric datasets are hard to create
A useful dataset for training a morphing attack detector needs to contain both genuine (bona fide) facial images and morphed images — lots of them, across a wide range of morphing techniques, spanning diverse demographics, with realistic variation in image quality and capture conditions. Biometric data is special category personal data under GDPR, requiring explicit consent and careful data governance.
The evaluation platform challenge
Beyond training data, the field needs independent evaluation platforms — systems that allow researchers to submit algorithms and receive performance metrics on held-out test data that the algorithm developers have never seen. The Bologna Online Evaluation Platform (BOEP) provides exactly this for morphing attack detection. NIST runs similar evaluation programmes for face recognition (FRVT), face image quality (SIDD), and morphing attack detection (FATE-MORPH).
The ENCHANTER framework
The ENCHANTER framework, developed within the EINSTEIN project and released as open source, addresses the practical challenge of standardising dataset preparation pipelines — handling the heterogeneous formats, quality levels, and metadata conventions of different data sources. By reducing the engineering overhead of dataset preparation, ENCHANTER allows researchers to focus on algorithm development and makes it easier to reproduce and compare results across different groups.
Open, independent evaluation is how the field measures genuine scientific progress. A biometric algorithm can only be as good as the data it was trained on and the benchmark it was evaluated against.
© 2026 EINSTEIN Consortium. EINSTEIN is funded by the European Union’s Horizon Europe programme (GA No. 101121280) and by UKRI (IFS 10093453). Views expressed are those of the authors only. www.einstein-horizon.eu