One of my colleagues recently returned from a conference on government program integrity with an interesting anecdote. He recounted a vendor presentation where the speaker was touting a 52% accuracy rate in their fraud lead generation system. So… nearly half of the system’s leads generated false positives. Not so sure I’d brag about that.
High false positive rates lead to wasted investigative time and money and unwarranted intrusions into the lives of legitimate program beneficiaries and service providers. Ultimately, they lead to a lack of confidence in the system itself and investigators revert back to more manual detection methods. When one considers all the important services governments deliver and the immense political pressure they endure, this is obviously not acceptable.
Shortly after hearing this story, we were asked to respond to a question about false positive rates and any existing industry standards or even benchmarks. While every vendor, including Pondera, makes claims about our system efficacy, very few standards actually exist. Conversely, our clients (the government program administrators) generally are subject to improper payment standards placed on them by the federal government.
I think there is a great opportunity, even responsibility, for governments to create these standards. Fraud detection standards would challenge the vendor community to “put up or shut up”, leading to more innovation. They could also be adjusted as the standards are met and surpassed leading to constant improvement. And they would provide governments with a uniform method for measuring vendor performance.
It is true that fraud detection systems still rely on quality program data and can suffer from the old adage "garbage in, garbage out”. So government would still share in the responsibility of meeting any new standards. But clearly, there is more we can do. And this would benefit all parties involved… except, of course, the fraudsters.