Many media members and rights advocates seem to have a flawed understanding of what the report really indicates, but Grother emphasizes that NIST’s role is to serve its two stakeholder groups. “There was all this specificity that was missing in the discussion before we put out the report, so we tried to expose the landscape of things that could happen, even if we didn’t actually test them,” Grother adds. ‘Well, what do you mean? Which algorithm? Or which system?”įalse positives, false negatives, or failures to detect a person could have significantly different impacts on the effectiveness of an application, and outcomes for its users. “Often they would begin with a statement like ‘well, face recognition is biased.’ I would ask them for a qualification. During the development of the report that we wrote I’d given a number of interviews,” Grother recounts. “We did try to include some text in the report to sort of guide reporting on that by both the academic community who sometimes evaluate algorithms, in addition to developing algorithms, and to the press community. Some in the media, however, and the public by extension, had difficulty properly framing what the issue was that NIST was looking into. The prospective end-users group also has a high understanding of what the criteria and factors for effective biometric algorithms are, if not quite as deep as the developer stakeholder group, Grother says, and they are well-equipped to understand NIST reports. “Were the points raised by Georgetown, and less so by MIT, were they valid, were they substantive, were the effects large, were they small? They wanted insight, so they did the right thing by asking us to look at this, I think.” government stakeholders were doing the responsible thing of trying to understand what the technical information was on this topic,” Grother observes. Reports from academic institutions, however, brought it to greater attention among government customers of face recognition technology, and the associated policy makers, and motivated NIST to directly address the issue in a report released at the end of 2019. The agency seeks to serve stakeholders among the algorithm developers and their customers, so issues like alleged bias, or performance differences among different demographics, take greater prominence in testing and reports to reflect those stakeholder’s concerns, NIST Biometric Standards and Testing Lead Patrick Grother tells Biometric Update in an interview.ĭifferences in performance between different demographics have been known about for years, and was first looked at by NIST back in 2002.

Nist vulnerability database