Five Methods for Measuring Unobserved Events: A Case Study of Federal Law Enforcement

Submitted by on Wed, 10/24/2012 - 14:11
Reports
PDF icon Five Methods for Measuring Unobserved Events.pdf
John Whitley
This report describes the challenges of measuring unobserved events such as tax cheating, drug smuggling, or illegal immigration.

Measuring program performance is relatively straightforward in many areas of government, such as social services, visa processing, and air traffic control.  But there are instances where assessing performance and success is much harder.  One particularly difficult area involves law enforcement, where a key goal is to prevent or deter bad outcomes – which can often happen without the knowledge of law enforcement officials.

This report describes the challenges of measuring “unobserved” events – such as tax cheating, drug smuggling, or illegal immigration.  These differ from “infrequent” events (another measurement area where success is difficult to judge), such as a terrorist attack.

The author, John Whitley, is a former executive in the Department of Homeland Security, where he faced these challenges daily.  He reviews practical methods already in use in some government and law enforcement agencies to estimate “unobserved” federal crime rates and ways to measure them.  He notes that applying these more broadly at the federal level “could enable the types of performance management reforms that have revolutionized local law enforcement.”

A key insight offered by Dr. Whitley is that there is often “institutional resistance to diverting scarce resources from enforcement to data collection and measurement.”  But he says that when law enforcement leaders do make the leap, they quickly become champions of the use of data.  Local police departments across the country have begun to use these kinds of techniques to undertake what has become known as “predictive policing” – where police move officers to places where crime is predicted to occur rather than responding to crimes that have occurred. 

This report is one in a series by the IBM Center exploring more sophisticated approaches to analyzing and using performance information. Other reports on the use of analytics include: