Multiple Biometric Grand Challenge - Challenge Problem Structure Overview

Challenge Problem Structure Overview

The Multiple Biometric Grand Challenge is based on previous challenges directed by Dr. P. Jonathon Phillips. Specifically the Facial Recognition Grand Challenge (FRGC) and the Iris Challenge Evaluation (ICE 2005). The programmatic process of a Challenge Problem is as follows. The Challenge Team designs the protocols, challenge problems, prepares challenge infrastructure, and composes the necessary data sets. Organizations then sign licenses to receive the data and begin to develop technology (mostly computer algorithms) in an attempt to solve the various challenges laid out by the Challenge Team. To advance and inform the various participants and interested parties the Team hosts workshops. The first workshop gives an overview of the challenge and introduces the first set of challenge problems (typically referred to as Version 1). The data sets are then released to participating organizations who develop their algorithms and submit self reported results back to the Challenge Team in the form of similarity matrices. The Team analyzes these results and then hosts another workshop. At the 2nd Workshop the Challenge Team reports the results from Challenge Version 1 and releases the Challenge Version 2. The cycle is repeated, finishing with a final workshop. At this stage the Participants are requested to submit not their self reported results, but the actual executables (or SDKs) to their algorithms. The Challenge Team then runs these algorithms through a battery of tests on large sequestered datasets. This phase ultimately determines the performance levels of the participant’s algorithms. A final report is issued by the Team which is used by Industries and Governments to determine the actual state of the art in a given field and to provide participating organizations a basis for showing their performance within that field.