3 August The Most Important System Feature August 3, 2016By Jon Coss - Blog Manager General investigators, built by investigators for investigators, fraud, intuitive system 0 I believe that simple things can make a big difference. This week, for example, I went through self-checkout at a local grocery store and the keypad gave me the options of “debit card” or “all other tenders” to complete my transaction. “All other tenders”—who talks like that? No doubt there was a group of people that decided that, technically, “tenders” was the best word to cover all the other options. It doesn’t really matter that it makes the system more confusing. That’s my problem.Software systems suffer from this problem perhaps more than any other consumer product. I remember the old joke about having to go to the “start” button to stop the computer. This still happens, despite the fact that experience has shown us that the single most important feature contributing to the success of software is usability.Put simply, even the most powerful system is completely worthless if people can’t figure out how to use it. My own brother discovered this when he recently decided to switch from the iPhone to an Android phone for the additional capabilities. Not a very technical person, he quickly switched back complaining that he was utterly confused by the “full fledged computer” he was carrying around in his pocket.At Pondera, we make the claim that our system is “built by investigators, for investigators.” And it’s true. Our most important design principle is to “mask” the underlying complexity of the system and provide analysts and investigators with an intuitive system that works the way they do. Technical people can’t do this. Data scientists can’t do this. Only investigators can do this. That’s why we hire them and task them with our most important work. Related Posts A Reminder of Government’s Important Role in Helping Americans Regular readers of this blog know that we often focus our comments on the fraud and abuses in government subsidy programs. Our intent is almost always to point out solutions that can make these programs more effective, rather than to question the existence of the programs altogether. In fact, most Pondera employees worked for government agencies in the past and recognize the good that many programs deliver.The recent fires in California provide an example of government programs helping Americans in need. The fires, which tore through California’s wine country, including Napa and Sonoma, claimed over 40 lives and 7,000 structures, and displaced nearly 100,000 people. Entire neighborhoods were destroyed leaving families scrambling for shelter and food.A number of government agencies sprang into action to help those affected by the disaster. While most Americans are somewhat familiar with, for example, how FEMA responds to disasters, other efforts are less obvious but very important.For example, I took note of the combined efforts of the United States Department of Agriculture (USDA) and the California Department of Social Services (CDSS) to provide emergency food assistance to local residents. Using the Disaster Supplemental Nutrition Assistance Program (D-SNAP), CDSS offered a month of food benefits to those who qualify. This is in addition to other waivers such as allowing existing SNAP beneficiaries to purchase hot foods (many lost their homes and by extension their ability to prepare foods).Events like the fires in California, while heartbreaking, also provide us with a reminder of the important work done by federal and state government agencies. This is also a time to remember the important work that agencies do throughout the year to support Americans who truly need our assistance. We believe our job at Pondera is to help governments deliver better services by driving fraud, waste, and abuse out of their programs. Why It’s Important To Be Nimble When Fighting Fraud Fighting fraud is an interesting challenge. When an organization shuts down a fraud scheme, bad actors don’t suddenly become good citizens. Rather, they evolve. They evolve by designing and testing new methods that, when they succeed, are exploited until they too are detected and shut down. In fact, many of the more enterprising fraudsters proactively try out new methods to avoid any “breaks in revenue”. This constant game of “whack-a-mole” is well known to program integrity staffs.This ever-changing nature of fraud schemes demands an ever-changing detection system. If an organization’s fraud detection methods look more or less the same as they did the year before, I can almost guarantee you that their fraud rates have returned to the levels prior to the introduction of the detection systems. Bad actors will simply find another way to conduct their business.For these reasons, it is important to be nimble in fighting fraud. Complex technologies and massive data warehouses will lose effectiveness over time. If it takes a year to negotiate a change order, to deploy new technologies, or to integrate new data sources, literally billions of dollars in improper payments can be lost. The trick is to stay even with or just a “step” behind the bad actors, not miles behind.While all state agencies want to choose the right partner to help them with their fraud detection efforts, I would challenge them to ask themselves the following questions: How easy is the company to work with? How willing will they be to incorporate new technologies (even if not their own)? How important is fighting government fraud to their company versus everything else they do? How passionate are they about the problem? Honest answers to these questions will reveal a lot about the future success of your efforts. Fraud, Waste, and Abuse Standards One of my colleagues recently returned from a conference on government program integrity with an interesting anecdote. He recounted a vendor presentation where the speaker was touting a 52% accuracy rate in their fraud lead generation system. So… nearly half of the system’s leads generated false positives. Not so sure I’d brag about that.High false positive rates lead to wasted investigative time and money and unwarranted intrusions into the lives of legitimate program beneficiaries and service providers. Ultimately, they lead to a lack of confidence in the system itself and investigators revert back to more manual detection methods. When one considers all the important services governments deliver and the immense political pressure they endure, this is obviously not acceptable.Shortly after hearing this story, we were asked to respond to a question about false positive rates and any existing industry standards or even benchmarks. While every vendor, including Pondera, makes claims about our system efficacy, very few standards actually exist. Conversely, our clients (the government program administrators) generally are subject to improper payment standards placed on them by the federal government.I think there is a great opportunity, even responsibility, for governments to create these standards. Fraud detection standards would challenge the vendor community to “put up or shut up”, leading to more innovation. They could also be adjusted as the standards are met and surpassed leading to constant improvement. And they would provide governments with a uniform method for measuring vendor performance.It is true that fraud detection systems still rely on quality program data and can suffer from the old adage "garbage in, garbage out”. So government would still share in the responsibility of meeting any new standards. But clearly, there is more we can do. And this would benefit all parties involved… except, of course, the fraudsters. Learning from Antivirus Software Almost everyone is familiar with antivirus software. Not everyone is familiar with how it works though. Even fewer have examined how we can apply the way antivirus software works to combat fraud. I believe that there are important lessons here which can improve our approach to fraud detection and prevention.At a high level, antivirus software performs two important functions prior to opening a file on your computer: 1) It compares the file to known viruses and other forms of malware, and 2) It checks the file for suspicious code which may indicate a new, previously unknown virus.The first function depends on a network of users willing to share known viruses and a system that is able to collect the virus data, design a fix, and disseminate the fix to other users prior to them being infected. The second function depends on heuristic programmers that can design systems to learn and even anticipate potential problems. Working together, this is one of the most effective ways to address the constantly changing nature of Internet malware.Government fraud prevention, when done properly, works in a very similar manner. By examining known bad actors, bad transactions, and bad behaviors, systems can quickly compare ongoing program data to identify suspect transactions. Modern fraud detection systems also include predictive algorithms that can detect anomalies, trends, patterns, and clusters that may indicate fraud.Unfortunately, many governments are unable, or unwilling, to share data. This limits the “network” effect that antivirus software uses so effectively. If more states and programs shared fraud schemes and findings, the library of known bad actors and methods could detect fraud and prevent it from moving from state to state and program to program.The good news is a number of states are moving toward state-wide fraud prevention efforts and a number of government subsidy programs are moving toward cross-state fraud prevention efforts. I am confident that the future success of these efforts will promote additional sharing, leading to a larger network, and more efficient governments. Pondera’s High Potential Leader: Amanda Huston Established in 1636, Harvard University is the United States' oldest institution of higher learning and one of the world's most prestigious universities. A couple of weeks ago, I had the remarkable opportunity to participate in the Harvard Business School Executive Education Program for High Potential Leaders. To step foot on this amazing campus filled with brick buildings plush with deep green climbing ivy, you almost immediately feel like you are part of something special (or perhaps inside some Matt Damon/Ben Affleck movie). Stepping foot in the state-of-the-art classroom with the instructor "pit" in the center, surrounded by 100 of the world's most talented and up-and-coming leaders, I wondered if I fit in this group or would have any common ground. My learning group, a smaller team designed to facilitate debate and discussion on assigned topics, included eight talented young professionals; only two originally from the United States. They represented a variety of industries, none of which had anything to do with mine. What I learned by working with this group, is that despite my initial hesitation, we were far more alike than I could have imagined. No matter their business, job title, or Country of operation, we faced so many of the same challenges and experiences in our professional lives. During one group activity, I began to think about how this applied to the clientele I serve at Pondera. Whether it's a small State unemployment program or the Nation's largest Medicaid program, these teams of dedicated professionals face so many of the same challenges and share similar experiences. Perhaps, I could bring them together through the Pondera client network and facilitate cross-state, cross-program sharing and learning. My brain was really starting to kick into high gear now. Reflecting back on my time at Harvard, I decided to focus on the key ways I could translate my experience into benefit for my company and clients. I decided upon three themes: Bold, passionate, inspiring leaders can change everything. No matter if you are managing financial accounts worth billions or a Government employee overseeing a Federal entitlement program, the culture created from these kinds of leaders brings success to the whole organization. Skills can be taught, management can be improved, but make no mistake, there is no substitute for extraordinary leadership. We must find these leaders, and then cultivate and cherish them. Networks are critical to continued learning and success; make time to grow and nurture yours. Your network could be persons within or outside of your organization, family, friends, peers, professional mentors, etc. Networks serve as a vibrant source of creative energy, partnership, and may just offer the solution to whatever challenge you or your organization is facing. Make time in your daily grind to have a coffee, make a quick call, or even share a meal with key persons in your network. Always be willing to adapt and evolve or be prepared for extinction. This is especially true in leading innovation, particularly in the data analytics arena. Fraud schemes change, data sources emerge, programs transform. At Pondera, we can never get comfortable or diminish our aggressive pursuit to lead the way. Governments must embrace the "information age" and transform their processes, modernize their programs, and challenge the status quo. Graduation The False Positive Problem Last year, 60 Minutes did a segment on the impact of errors in the Social Security Administration’s Master Death File—a database that stores dates of death for Americans. The system stores 86 million records, and despite all best efforts, it still has some issues.60 Minutes pointed out that errors in the system contribute to millions of dollars in improper payments each year. After all, the system would seem to indicate over 6.5 million Americans over the age of 111 when, in fact, there are probably fewer than 100. On the other hand, false positives where people find themselves mistakenly placed on the list, lead to nightmarish scenarios for obtaining loans, opening bank accounts, and other everyday tasks.This story demonstrates one of the largest challenges for government agencies: how to use imperfect data sources to minimize fraud, waste, and abuse while also not “harassing” legitimate people and businesses. And unlike a private business that may view a false positive as an inconvenience (who hasn’t had to call their credit card company to say that “yes, that large ice cream purchase was legitimate”), government officials are severely criticized when they act on false positives. In effect, they are criticized for not acting and they are criticized for acting.Pondera suggests that governments mitigate the effects of false positives by using composite indicators that draw information from multiple sources—both simple data matches like the Master Death File and more complex behavioral sources. For example, a Medicaid investigator would feel much more confident looking into a person who not only shows up on the Master Death File, but also appears to be traveling 100 miles for 20-minute doctor appointments, receives highly unusual (and expensive) procedures for their apparent diagnosis, and often sees two doctors in distant cities on the same day.Anyone who has worked for or with government program integrity units understands the unique pressures they face. Combining available data sources with intelligent analytics can go a long way toward helping them investigate the right cases while not interfering with program delivery. Comment (0) Comments are closed.