Probable Causation

View Original

Episode 66: Aaron Chalfin

Jennifer Doleac

Aaron Chalfin

Aaron Chalfin is an Assistant Professor of Criminology at the University of Pennsylvania.

Date: February 1, 2022

A transcript of this episode is available here.

See this content in the original post

Episode Details:

In this episode, we discuss Prof. Chalfin's work on the professional motivations of police officers:

“The Professional Motivations of Police Officers” by Aaron Chalfin and Felipe Goncalves.

OTHER RESEARCH WE DISCUSS IN THIS EPISODE:

TRANSCRIPT OF THIS EPISODE:

 

Jennifer [00:00:08] Hello and welcome to Probable Causation, a show about law, economics and crime. I'm your host, Jennifer Doleac at Texas A&M University, where I'm an Economics Professor and the Director of the Justice Tech Lab. 

 

Jennifer [00:00:19] My guest this week is Aaron Chalfin. Aaron is an Assistant Professor of Criminology at the University of Pennsylvania. Aaron, welcome back to the show.

 

Aaron [00:00:28] Thanks, Jen. A pleasure once again to be here. 

 

Jennifer [00:00:31] Today, we're going to talk about your research on how police officers respond to incentives. But before we get into that, could you tell us about your research expertize and how you became interested in this topic? 

 

Aaron [00:00:42] Sure. So I am a professor of criminology, and one of the topics that I spend the most time studying within criminology is policing. And the focus of a lot of my earlier work has been on trying to understand whether investments in policing have an impact on public safety. So when cities expand the size of their police forces, what happens to crime and violence, or how did the benefits and the cost of policing vary by race? I've also written about shifts in policing strategy and the effects of that sort of thing on public safety. So really, most of my work studies the social and economic returns of investments in police, but you can't really study the effects of police without at some point developing an interest in the police officers themselves and trying to dig a little bit deeper and understand who those people are. 

 

Aaron [00:01:33] And let me maybe say just a little bit more about that. So there's around 700,000 municipal police officers in the U.S., so maybe one officer around every 300 adults living in a big city or medium sized city and cops are basically regular people who are given six months of training, which I think is less time than it would take to become a cosmetologist. And then they're given a badge and a gun, and they have a mandate to enforce the law. And we ask police to do a lot of things. We ask them to prevent crime and violence, to promote traffic safety, to be first responders to medical emergencies, to deal with people having mental health crises, to solve homelessness, to comfort crime victims and so on and so on and so on. Right. We ask them to do a lot. And if you think about it, the stakes are really incredibly high. So if a police officer makes a mistake and fails to make an arres that should have been made, then you have an offender on the street who is free to harm someone else. And if someone's arrested by the police, that person can lose his freedom. Maybe also his job, his marriage and most of his human capital, so an arrest is really a weighty thing. And so for a lot of people, right, when you screw up at work or you make a bad decision, it doesn't carry life-changing consequences. But if you're a cop, it's just sort of part of the job. 

 

Aaron [00:02:55] The other thing that fascinates me about policing and the motivations of police officers and this goes along with the high stakes, is the extent to which outcomes in our criminal justice system more broadly are just always going to be filtered through the informal decisions that are made on the street by cops. Right. So just to give some examples, like police officers aren't lawyers, but they're often in a position to decide what's meant by legal terms like reasonable suspicion or probable cause. And police officers aren't judges, but they often are going to be deciding about what behaviors are going to be punished. And so police really have a ton of discretion. The discretion that they have can be a bad thing if it makes the law more arbitrary or capricious or maybe discriminatory. But it can also be a good thing where police can empower communities by enforcing the law in ways that solves social problems and accords with an ordinary person's sense of justice. And so, you know, to sort of wrap this up I think it's really important to understand how police officers are motivated, what they care about and what their values are. Do police have what we might think of as pro-social motivations towards their work, perhaps in the same way that, say, a child abuse advocate might have or a social worker? Are police corrupted by financial incentives that they face on the job because sometimes they definitely do face incentives like this. And so ultimately our high level motivating question here is what makes these guys tick. 

 

Jennifer [00:04:25] Yeah. So the paper we are going to talk about today is called "The Professional Motivations of Police Officers." It's coauthored with Felipe Goncalves. And in it, you consider whether police officers arrest activities in particular are driven by financial incentives. So what do we know about how police behavior responds to financial incentives more broadly? 

 

Aaron [00:04:45] Yeah. So broadly speaking, there's I think at this point, quite a bit of evidence that the work of law enforcement can be distorted by financial incentives. So in the U.S., which is the place that I'm most familiar with, police departments tend to change their enforcement of traffic tickets when fiscal conditions are tight. They issue more tickets in particular to out-of-state drivers, which is exactly what you might think that they would do if they cared about their financial incentives. Drug arrests rise when asset forfeiture laws empower police to recover a higher share of seized assets and drug investigations. Thinking back to 2015, when the US Department of Justice issued its report about policing in Ferguson, Missouri, this is in response to the killing of Michael Brown there, it noted a pattern of excessive policing, in particular in minority communities that seemed designed to raise revenues. So I think at the institutional level, there's a lot of evidence that financial incentives matter, that organizations respond to the incentives that policymakers provide for them. 

 

Aaron [00:05:47] But I don't think we know a whole lot about the motivations of police officers themselves. So we have some information. There's a, I think, 2006 paper by Alex Mas that shows that when police unions lose an arbitration case, police activity slows down, perhaps in protest. There's two recent papers one by John Mummolo, another by Bocar Ba and Roman Rivera that suggests that police behavior was actually very responsive to things like managerial oversight and messaging from the police union. So we know some things along those lines. There's also some ethnographic literature on this question. A favorite of mine is a book by Edith Linn, who's now a professor at the John Jay College in New York City, but in a former life, she was a New York City police officer. And so she talks about some of the stuff that matters in arrest decisions. There really isn't a whole lot of quantitative evidence out there, especially on the degree to which individual police officers are motivated by money and whether money might distort the decisions they make on the job. 

 

Jennifer [00:06:50] I gather that the motivation, or at least one motivation for this paper, comes from what's often called the collars for dollars hypothesis, that is the possibility that police officers make more arrests late in their shift because they want to earn overtime pay. So tell us a bit about police overtime and why an arrest might trigger it?

 

Aaron [00:07:10] Yes, that's right. And to give you a little bit of a sense for how the sausage got made in writing this paper, when Felipe and I first started thinking about what became this project. We really I think we're narrowly interested in figuring out whether police make so-called collars for dollars. That's police lingo. What it refers to is whether police make late shift arrests, particularly low quality or maybe racially motivated arrests, late in their shift because they want to get overtime pay. So that's the question we started out trying to answer. And if you read popular news stories, you'll see a lot of articles about this, including some very recent ones. Those articles will often contain quotes from defense attorneys in particular who will say, like, "Look, I have a client who was arrested wrongfully or unfairly. It occurred at the end of the officer's shift, and it was really clear that reason this arrest was made is because the officer just wanted to collect overtime pay. This is wrong." Right. And if you stop and think about an allegation like that, it's really incredibly serious, right. The idea that a police officer might arrest someone wrongly or even rightly, for that matter, just so they can supplement their pay is really an odious practice. Police are public servants. We entrust them to uphold the law, and they're definitely not supposed to use their authority to promote their own interests. So that's a little bit of the background. 

 

Aaron [00:08:30] You also asked the overtime pay. In general, how do officers get access to that? And I'll give you the high level overview understanding that there's variation from city to city. We have thousands of police departments. So police officers, like most non managerial workers in the United States, are covered by the Fair Labor Standards Act, which is a federal law that specifies that work that's done outside of normal working hours, typically an eight-hour shift, is subject to a wage of time and a half - 150% of the normal wage. So this is a law that was actually passed back in 1938. A little bit of trivia. The original law was drafted six years earlier by Hugo Black, who at the time was a U.S. senator, is better known as later serving in the U.S. Supreme Court. And this is a law that exists today and really hasn't changed a lot in the many years since it was passed. And so this covers all police officers in the US. So when a police officer makes an arrest that requires his or her tour to be extended, there's this federal mandate that overtime wages have to be paid. An arrest takes a lot of time. Typically, it takes at least a few hours in officer's time to make even a fairly simple arrest. The process does vary a bit from city to city. In our setting, which is Dallas, Texas, what officers have to do is they have to transport the arrestee to central booking. They have to wait with the suspect while that person's fingerprinted and booked into jail. They have to fill out a bunch of paperwork. There could even be more to do. For example, if the suspect needs medical care or if it's a complicated arrest, then this can actually take a whole lot of time. With a few very minor exceptions, the work can't be delayed until the next day. It's got to be done immediately. So you can't just sort of say, "Well, I'm going to go home, and when I get into work tomorrow, I'll take care of that." So these arrests that are made later in the shift have a very high probability of leading to overtime pay. 

 

Jennifer [00:10:28] And overtime pay is, of course, not the only incentive that might drive police arrest decisions. So what else should we have in mind as possible reasons that a police officer might decide to make an arrest or not? 

 

Aaron [00:10:40] Yeah. So this is a great question, really interesting question, especially if you study police. So for starters, arrest decisions are governed by laws and also formal departmental policies. So ultimately, police have to follow the law. They're certainly supposed to. But while the laws are often binding on what police officers do on the job, there's just always a lot of discretion because police officers are imperfectly monitored. It's changing a little bit now with body worn cameras, but police make a lot of snap decisions that recognize that there's a lot of gray area in law. So with respect to exercising their discretion, police are always going to have their own view, as well as informal guidance from their department and their commanding officer about the social value of arrests. So is the arrest going to make the community safer? Is the arrest actually going to lead to a conviction? Will the person actually get punished or is it just going to be catch release? Does my department want me to make arrests like this one? Do people in the community want me to arrest a person like this? Did the suspect piss me off? Right. Did he talk back to me. There's all sorts of reasons for why some arrests are made, and ultimately, others aren't. One thing that I want to point out that's going on here in the background is that the formal incentives that officers face to make arrests are what Felipe and I call in the paper are very low power. And what we mean by that is that in most departments, including Dallas, which is again the setting for our study, you don't get paid and you don't get promoted for making more arrests or making better quality arrests. The only way to get paid more is to remain on the job and get promoted by rules negotiated with the police union. Promotions are based entirely on a civil service exam, so just being a good test taker is the thing that matters. And so the professional stakes for making an arrest are lower than many might think. There aren't career rewards, but there is this background discretion. 

 

Aaron [00:12:34] Now, with respect to making arrests that might lead to overtime pay, in addition to all the stuff I just mentioned, officers are also maybe going to be thinking about some particular private considerations. Maybe they're thinking about whether they're tired or not, or whether they have post-work social activities, a second job, childcare responsibilities, anything that they might have going on. And when you read ethnographic work about this, a truly shocking theme emerges that police officers are basically regular people like the rest of us and have a range of things going on their lives that can affect the way they do their jobs. And so overall, when I explain in officers late shift arrest decision to other academics I've been asking them to think about the following scenario. And so maybe this will resonate with at least some of your listeners. So let's say you've just gotten done teaching two three-hour statistics classes back to back and you're kind of tired. And just as you're packing up to leave your office, your department chair comes running in and says, You know what, I'll offer you one hundred and fifty dollars if you'll just spend another three hours tutoring, let's say some particularly uncooperative students in a basement classroom at the other end of campus. Like is that going to seem like a good deal to you? Answers are going to vary. But I think at a minimum, the answer to this question isn't entirely obvious. And so there's always going to be a lot going on here when you think about whether officers want to work overtime for their money or not. 

 

Jennifer [00:13:59] Okay, so despite lots of anecdotal evidence and those news articles and certainly public concern about this question, it's been really difficult to study the effect of personal financial incentives or any of these other incentives on police decisions. So why is this such a difficult question to answer? 

 

Aaron [00:14:17] Yeah, no. It is a hard question to answer, I think. And maybe one perspective on this is to think about what the ideal experiment would be. Let's say we could induce random variation in the financial incentive to make an arrest. So maybe at the beginning of an officer shift, let's say the officers are offered a different randomly assigned bounty for arrests made that day. And by varying that randomly assigned bounty, you can see how the volume of arrrest response. Right. That would be the ideal natural experiment. For obvious reasons and for good reasons, you can never do that. So what about a natural experiment? But here's the problem there just isn't a lot of variation in the incentive to make an arrest. As I mentioned a few minutes ago, officers don't have a lot of direct professional incentives to make an arrest. They're not paid a piece rate. They're not paid a bonus. And they're career tracks don't really depend so much on this. And so we're going to have to look for another source of variation. And what Felipe and I do is to leverage the natural experiment created by an officer's work shift. And what we do is we recognize that late shift arrests are far more likely to get that overtime pay than earlier shift arrests and therefore those arrests are worth more money. If you look at the number of overtime hours that's typical in our setting of Dallas, a late shift arrest is worth around $150 in extra overtime pay, which for comparison purposes is around 2/3 of an officer's gross daily wage, so it's a meaningful difference. 

 

Jennifer [00:15:48] And in addition to what you just described, which is basically an identification challenge here, I gather it's also just really hard to get the kind of data you need. I mean, in general, I think we don't have much data on what police are doing with their time or what their schedules look like and so on. It's just hard for research to get that. Can you talk a little bit about that, too? 

 

Aaron [00:16:07] Yeah. So this is absolutely right. You know, if you try to figure out how officers are spending their time, only very recently has that become more possible as departments have invested in better data gathering, better record keeping. Now we're starting to be able to track officers using GPS technology, but it really did take a fair amount of looking to find a city where we could get all the data that we needed and where they were very willing and graciously shared a lot of data with us and helped us to understand that data. So people have asked me, would it be easy to replicate this study elsewhere? And I say, well, there's a lot that you're going to need to go right. So a lot of ingredients. 

 

Jennifer [00:16:51] Yeah. Okay, so you're going to use the shift schedules and in particular overlapping police shifts as a natural experiment here to isolate the effect of shift hour from the time of day, which people might already be thinking might be a confounding factor. So in Dallas, Texas, officers work on staggered shifts that overlap rather than everyone just coming in and working 9:00 to 5:00, for instance. So what do the shifts schedules look like in Dallas and say more about why that's helpful for you? 

 

Aaron [00:17:19] Yeah. In Dallas, officers work these overlapping shifts, right, which means that in a given police command, there's always at least two sets of officers who are policing the same area of the city at the same time. But we're in different parts of the work shift. And the reason for this, I think, is fairly obvious that you don't want all of the officers going home at the same time just as new officers are coming on. In theory, offenders could exploit that, although there is some work that suggests in a different setting that they don't always do. But that's the reason for the staggered shifts. And so, for example, let's say that you and I are both Dallas police officers working in the same police division. So at three o'clock in the afternoon, you might be near the beginning of your shift, but I might be near the end of mine. And so even though we're facing, in theory, at least the same crime environment and the same level of demand for police services, we potentially face very different private incentives to make an arrest. And in a nutshell, that's the comparison that we're making to figure out if cops have a taste for these late shift arrests. So it's a really critical feature for us. If there weren't overlapping shifts, we would not be able to disambiguate between times of the day that have more arrest activity for any number of reasons and the importance of an officer shift, which again, is what we ultimately care about in this study. 

 

Jennifer [00:18:37] So exactly how will you use this shift schedules to measure the causal effect of financial incentives on arrests? Tell us a little bit about your empirical strategy here. 

 

Aaron [00:18:45] We address the importance of financial incentives and arrest decisions in two parts. So we begin by saying if there's any evidence that officers increase their arrest activity towards the end of their shift in response to the strong financial incentives that they have to do, so we're going to do that by comparing the two officers who again are working in the same place at the same time, but are at different points in their shift. That's going to help us understand how officers value their financial incentives vis a vis their non-work time. And next, we're going to study what happens when those financial incentives to make an arrest change. And that analysis is going to give us some insight into how officers trade off their financial incentives against their professional considerations, how much they care about their job. And so given that overtime pay is fixed by federal law, right, it's always time and a half. The question is how can we find something that changes the value of overtime pay? And so to do that, we're going to turn to a second natural experiment. And what we're going to use is the fact that officers sometimes moonlight in an off-duty job right after their police shifts. So maybe they're working security at Walmart. Maybe they're working security for one of Mark Cuban's swanky evening parties. And in those days, right, the value of a late shift arrest is just going to be quite a bit lower because the officers are going to get their overtime wage, which federal law mandates, but they're no longer going to receive any of their private wage. So the opportunity cost falls, and  that's really what we're going to be doing to sort all this out. 

 

Jennifer [00:20:16] Great. Okay. And then as we were discussing earlier, you need really detailed data for the study, much more detailed than researchers are usually able to obtain from police departments. And so you all searched around and were able to find this data in Dallas. Tell us about what you were able to get from that city. 

 

Aaron [00:20:32] Yeah. So we have really great data. I think I've listened to most of the Probable Causation episodes at this point, and everyone always says we have great data; no one ever says we have, you know, data that's about average. But no, I think we really do have very rich data. And what I mean by that is we can trace an event as it moves throughout the entire criminal justice system from start to finish. So for every 911 call, we know whether police made an arrest. We know what happened to that arrest downstream, whether it resulted in a conviction. Ultimately, if there was a criminal sentence imposed, we know what for and how long. We can link all of this back to the original officers who took the call. And critically, we also have data on officers assigned shifts, as well as data on off duty work that they performed because they have to report that to the police department. It's actually a very serious thing if you don't report that. So we know a whole lot about an officer's workday and what they've been up to. 

 

Jennifer [00:21:28] So tell us about your sample. What did the officers in your sample look like? How often do they work overtime and how much they earn? That sort of thing. 

 

Aaron [00:21:35] Yeah. So Dallas is, I think at this point, although the 2020 Census scrambled just a little bit, I think it's the eighth largest in the United States. It's home to 1.3 million people. Dallas employs around 3,500 cops, so it's one of the largest police forces in the U.S. and it's a diverse police force. Half the officers are nonwhite, including a quarter who are black and 20 percent who are Hispanic. Those demographics are actually very similar to those of the Dallas Fort Worth area in general, which is interesting and true of actually a lot of large city departments. Policing is a more diverse profession than I think many people believe it to be. Sixteen percent of the force are women. On average, our officers have 10 years of experience. So these are people who stay in the job a long time. With respect to pay mean base pay is $65,000 for officers in our sample, and they earn another $6,000 in overtime pay. So overtime pay is around nine percent of an officers base wage. 

 

Jennifer [00:22:39] If I remember correctly, there's a lot of variation there. Right. So like, it's not like everybody is, you know, doing a little bit of overtime over the course of the year. You've got like some people doing none and then some people who are raking in quite a bit of money on the overtime. Was that right? 

 

Aaron [00:22:53] Yeah, that's right. And thanks for actually noting that because I think I would have forgotten to say that there is a lot of variability many officers never take over time. But we have a couple of officers who make as much as 80 or $90,000 a year or in a few cases, a little bit more. And that's actually important, I think, because it suggests that there aren't informal caps on how much overtime you can take. When we've called around and spoken to people in Dallas. We're told that there really are no informal caps. No one is tracking how much overtime pay officers are taking as long as everything's being reported correctly. So it's not something that is constrained by things that are outside an officer's own preferences. 

 

Jennifer [00:23:36] So what outcome measures are you going to focus on here? 

 

Aaron [00:23:38] We focus on two main outcomes. So first, we study the incidence of arrests. We ask how does the number of arrests that an officer makes vary throughout his or her work shift? And next, recognizing that the collars for dollars story is not just about the number of arrests that are made, but also the quality of those arrests. We have some measures of arrest quality, and there's no perfect measure of arrest quality. I think that's always in some sense going to be a political question or in the eyes of the beholder, but we have two proxies that I think are reasonable. So first, we study whether an arrest led to a criminal conviction for something that's at least as serious as a misdemeanor offense, so not, something very, very minor like a violation. So in other words, was the arrest ultimately sustained by a criminal court process? And second, we're going to consider whether an arrest received a criminal sentence in either jail or prison. So in other words, did the courts consider the offense to be serious enough to merit the use of correctional resources? So two different measures of arrest quality both admittedly imperfect but which capture a couple of different dimensions of quality. 

 

Jennifer [00:24:45] And what types of crimes are people typically being arrested for? 

 

Aaron [00:24:49] Yeah. So there are five types of crimes that make up half of the arrests in Dallas, and this is, I think, very similar in every city I've ever seen. It looks just like this. So they're disorderly conduct, assault, drug charges, retail theft, shoplifting and public intoxication. So that makes up the lion's share. 

 

Jennifer [00:25:09] Okay, great. So let's talk about the results. First, you consider the effect of shift hour on the frequency and quality of arrests. What do you find? 

 

Aaron [00:25:18] Yeah. So despite the fact that these later shift arrests are more valuable with respect to financial enumeration, we actually see that after peaking a couple of hours of an officer's shift, arrests fall pretty steeply thereafter. So from peak arrest activity falls by around a third. Now, with respect to arrest quality, the rate at which arrests lead to a conviction or criminal sense actually increases throughout the workday. So it increases by about a quarter for the probability of a conviction and by about 40 percent for the probability of a sentence. So what that indicates to us is that officers are raising their threshold to make those late shift arrests, not lowering their threshold. We see these effects for serious crimes, as well as less serious crimes. The effect is a little bit bigger for less serious crimes, which makes sense. And then notably, we don't see any evidence of racial disparities that show up at the end of the workday. It's not like the late shift arrests are more likely to accrue to people who are black or Hispanic. 

 

Jennifer [00:26:16] So this is the opposite of the collars for dollars story. Is that right? 

 

Aaron [00:26:20] Exactly the opposite, right. We're seeing strong evidence that officers are raising that threshold at the end of the shift in making arrests, not lowering it. And what this tells us, at least if you believe our identifying assumptions, we can talk more about that, officers value their leisure time quite a bit at the margin, at least more so than the overtime pay, but they would have received. 

 

Jennifer [00:26:40] Yeah, they're tired. It's a hard job. They just one wanna go home, basically. Okay, so you also consider the effect of making an arrest conditional on taking a call for service, so that is responding to a 911 call. So why do you do this and what do you find? 

 

Aaron [00:26:56] Yes, that's right. And what we do in a separate analysis, right, is instead of considering all the arrests that officers make, we focus specifically on emergency calls for service that officers are routed to. So these are 911 calls made by citizens, and the closest officers are routed to that call. And what we do is we study the probability that a call for service leads to an arrest and how that changes throughout the officers shift, and it's actually a really, really important part of the paper. And the reason why it's so important is that the results I just told you about rest on some really, really big assumptions. Assumptions that I think everyone should be skeptical about, which is that the pattern and arrest activity that we observe is going to be due to the officer's underlying preferences and not some other constraint or some other feature of police work that operates outside of preferences. 

 

Aaron [00:27:48] And how can we sort of address that? Well, as it turns out, by focusing on calls for service we can actually address a lot of alternative explanations for why arrest activity might fall at the end of the day. I'll mention a couple of these concerns for brevity, but it actually addresses even more than what I'll mention right now. So first, I think a common concern right when thinking about this pattern would be that officers are simply being taken out of circulation when they make earlier arrests and never return to patrol. So if that's the case, then of course, you would find that arrests are lower at the end of the shift because the officers are just being incapacitated by earlier behavior. 

 

Aaron [00:28:25] Second, this is related. You might worry that 911 call operators are simply being smart and routing fewer officers to calls at the end of their shift. That actually would make a ton of sense for a police department that's seeking to control the use of overtime pay. And so that could also explain this finding. And then the other thing you might think about, and this is something that comes up a lot in police ethnography is the practice of so-called arrest trading, whereby a late shift officer hands off an arrest to an officer who's at the beginning of his or her shift. And maybe that could be happening. And we can actually address all of these stories by focusing on service calls, so officers who respond to service calls are by definition in circulation, right. Otherwise, they wouldn't be routed to a call. So the results aren't explained by the incapacitation of earlier arrests or strategic behavior by 911 operators. And then by studying whether a service call leads to arrest we're baking in any arrest trading that might be happening behind the scenes that we can't directly observe. And so when we run the same analysis at the service call level, the same familiar pattern emerges that when a call comes in at the end of the officer's shift, an arrest is about 25 percent less likely. So it's not a third less likely. It's 25 percent less likely, which suggests that some of the stuff that I mentioned matters, but it doesn't erase the story. 

 

Jennifer [00:29:45] So like the good social scientists you are, you dig a little bit more and you consider the robustness of your results and the extent to which they vary across groups in several ways. Are there any analyses there that you want to highlight that you find particularly valuable? 

 

Aaron [00:29:58] Yeah. So the paper has a lot of appendix figures. There's even more sitting in our Dropbox folder that we're omitting simply because we we're not trying to write a Tolstoy novel, right. We don't want to overwhelm readers with, you know, stuff that only four people care about. But basically, let me just sort of summarize this by saying that we haven't been able to do anything that makes the main results about the time path of arrests go away. We varied all of the fixed effects that we used in the model a million different ways. We have a couple of balanced tests that we employ to check for, whether early in late shift officers truly are facing the same crime environment. And while I think the results are subject to different interpretations, the results are very, very robust. It's as robust as anything that I've worked on in my career. It just you really can't do anything to destroy it. 

 

Jennifer [00:30:50] Okay. So as you mentioned earlier, you also use the fact that some officers take off duty work before or after their shifts. So tell us a bit more about how often officers do this and remind us why it's useful in this analysis. 

 

Aaron [00:31:05] Yeah, and thanks for the reminder, because I was just about to jump into the meat. So thus far, what I've shared with you is that police officers are considerably less likely to make late shift arrests. And we're arguing that this is evidence that officers placed a lot of value on their leisure activities relative to their financial rewards, at least at this particular margin. So that's valuable, I think. But what that natural experiment doesn't allow us to do is to separately identify the relative importance of financial rewards versus the non-financial components of work. So basically, do officers value doing their jobs well? And to study that, we need something that actually shifts the cost of making the rest. Now, unfortunately, in the US, or fortunately, depending on your perspective, the rate of overtime pay is fixed by federal law. And so there's no direct variation that we can exploit that changes the value of a late shift arrest. So instead, we're going to turn to a different sort of institutional arrangement that I referenced maybe 15, 20 minutes ago that does have the effect of shifting the opportunity cost of these late shift arrests. And this is off-duty employment performed for private businesses. It's actually around half of Dallas police officers do some sort of off-duty work during our study period, and it's a pretty common thing. 

 

Aaron [00:32:24] And so here's what we're going to do with this. So on days when an officer is scheduled to work an off duty shift after her police shift, the value of overtime work is just lower right. Again, get your overtime wage from the police department, but you forgo the wage that you would receive. And given that the average off-duty shift is around five hours, the change in opportunity cost is actually quite large. And just to be really clear, the intuition here is going to be the way we're going to use this to understand how responsive officers are to financial incentives is we're going to reason that if officers further reduce their late shift arrest activity on days when they're scheduled to work an off duty job after their police shift that would mean that they are responding to their financial incentives. That would mean that their private financial motivations to go out and earn their private wage are sufficient to distort their work activities. And so that's the underlying logic here. 

 

Jennifer [00:33:22] Okay, so what do you find is the effect of off duty work on arrests? 

 

Aaron [00:33:27] Yeah. So what we find is that on days in which an officer is scheduled to work an off duty job after their police shift, the late shift arrests fall by an additional four percentage points on top of the normal decline that we observe of around 25 to 30 percent. So because late shift arrests decline more in these days than they do on other days, we see this as evidence that officers are in fact responding to their private financial incentives. So an officer who didn't value their private financial incentives at the margin, that officer would not have changed their late shift arrest activity on these days. Now, importantly, we don't see changes in late shift arrest activity on days where the officers working an off duty job before their police shift. And what that tells us is that the slowdown in arrest activity towards the end of an officer shift is probably not driven specifically by fatigue. The officers are tired. They're just not as good at identifying people to arrest. It doesn't seem like it's something like that. It seems to be much more likely to be driven by preferences. 

 

Jennifer [00:34:31] Interesting. Okay. And then you move beyond the analysis of this natural experiment to construct and estimate a model. So tell us why you do this and what's the general intuition for what the model does? 

 

Aaron [00:34:44] Yeah. So a model is not always easy to explain in a podcast, but I'm going to do my best. I'll just keep things brief to give a sense for the high level reason for doing this. So we use a behavioral model for two reasons. So first, the model is going to help us to understand our findings in a deeper way. So I just told you that officers are responsive to financial incentives. They do reduce that late shift arrest activity on days where they're set to moonlight an off-duty job. The models going to allow us to sort out whether that's a big response or a small response, it's going to allow us to come up with an estimate of how much money we think it would take to get an officer to knowingly make a corrupt decisions, knowingly make a wrong arrest just because they want to get more money. The second thing the model is going to help to do is it's going to help us to clarify some of the underlying dynamics of arrest decisions that a normal statistical analysis might miss and could even lead to a misleading conclusion. So I'll just describe the model again at very high level. It's a model of type one and type two errors in making arrest decisions. So by type one and type two errors, I mean that officers can incorrectly arrest an innocent person or they can incorrectly fail to arrest a guilty person. Those are errors that an officer who cares about his or her job is going to want to minimize. So officers are going to make these decisions under some uncertainty, but they're going to have some information about an individual's guilt and they're going to make an arrest, according to some rational built probability. 

 

Aaron [00:36:21] And there's going to be two added considerations here in the mix. The first consideration is that any arrest you make is going to mean that you're forgoing the option value of remaining on the street and making a later arrest that might have been a better arrest. And second, that if you make an arrest, it might require overtime work, which might require that you have to re-juggle your life now. And so when we write down our simple model and we've said it to our data, we find that the model fits the data very well. We recover all of the essential dynamics that are revealed in our empirical analysis. 

 

Jennifer [00:36:56] Okay, and then what do you find in terms of how officers appear to respond to financial incentives versus other things, like the likelihood that a suspect's guilty? 

 

Aaron [00:37:06] Yeah. So what we do is we use our model to calculate the amount of extra pay that you'd have to give an officer to arrest someone who's definitely innocent or to fail to arrest someone who's definitely guilty, right. So this would be clearly corrupted decisions. Or, you know, this would be a distortion of what police should be doing. And to be clear, officers' responsiveness to financial incentives, the way we estimate this right, linking back to our empirical model, we're looking back to see how responsive arrest decisions were to off-duty work. That was something that shifted the financial incentives to make an arrest. And the estimates from our model are very high and we take several thousand dollars, we think, to get an officer to make a decision that they know is a wrong decision, which I think is the good news in all of this. 

 

Jennifer [00:37:54] Yeah. And relative to the overtime pay, which is somewhere in the ballpark of what? 

 

Aaron [00:37:59] 150 bucks. Yeah, it would not be enough to distort the behavior of all but a few officers is what we would find. 

 

Jennifer [00:38:07] And then you also use your model estimates to predict what would happen if officers were what you call purely altruistic. So what does that mean in this setting and what do you find? 

 

Aaron [00:38:17] Yeah. So as I sort of alluded to this, the model is useful because it reveals some behavioral dynamics that I think are not obvious from using a standard statistical analysis. So in an ordinary regression model of officer arrest activity, a pattern of increasing arrests throughout the day would be taken as evidence that officers are engaging in collars for dollars that they're making late shift arrests in exchange for overtime pay. When we began this project, that was how we would have interpreted a finding like that. But when we started to think about this a little bit more deeply, and running the model helped us to do this, we realized that the model makes clear that that sort of behavior increasing your arrest activity throughout the shift is actually totally consistent with what an officer who's altruistic or an officer who, to be more precise, is over time indifferent would do. Like that officer, an altruistic officer is going to ramp up their arrest activity throughout the shift. And the reason why is that this over time in different officer is going to be raising her arrest threshold throughout the shift because the declining option value of time, right as the day wears on, the probability of making even better arrest declines. And so at some point at the end of your shift, you're like, well, this is as good as it's going to get, so I'm going to make the arrest. And so in the simple empirical model, an officer who behaves that way is going to look like they're making collars for dollars but is actually going to be purely overtime different. And so what we're seeing this pattern of declining arrest activity is not consistent with something like altruism. 

 

Jennifer [00:39:53] Yeah, and just to make sure we're on the same page here. So the story you're - you have in mind is I'm an officer and I'm just starting my shift and like the minute I walk out the door, I see someone who's drunk on the sidewalk and like, I could arrest that person. But if I arrest that person, that I might miss that on the next block, someone's breaking into a car or holding up a bank or something like that. So if I arrest that first person for the really minor offense, then I'm sitting inside of my desk doing paperwork for four hours, and I don't have the opportunity to do other things I might feel better about and might know are more useful. And so that's what you're talking about in terms of like the opportunity cost of time. And so as the shift is going on in the seventh hour of my shift at that point, there's very little possibility that I'm going to miss out on bigger crimes if I arrest this person now. And so I'd be more likely to arrest them just because of that. Am I getting that right? 

 

Aaron [00:40:45] You're getting that exactly right, and I think that was actually probably more helpful, more clear than how I explained it. Yeah, that's exactly right. 

 

Jennifer [00:40:53] All right. So what are the policy implications of these results? What should policymakers and practitioners take away from all this? 

 

Aaron [00:41:01] Right, so sort of linking back to how I tried to frame the paper at the beginning of our conversation. I think the big question here is does this mean that officers are professionally motivated? Does this mean that officers are responsive to financial incentives? What do officers value? I think the evidence is actually kind of mixed here, right. So on the one hand, we observe clear evidence for what I think a reasonable person could think of as shirking. Right arrests fall later in the shift, even for serious crimes, because officers clearly value their leisure time quite a bit. That's probably not what many people would want to see from their police officers. But you know, this finding might actually explain why police departments in the past have explicitly incentivized officers to make lots and lots of stops and lots and lots of arrests, right. Sort of what broken windows policing morphed into. I think the reason why this happened is that senior management police departments know police officers better than journalists do, and they understand that officers might actually be tempted to shirk because they want to go home at the end of the day. Now, I'm not saying that police departments should have been doing this. I think there's a lot of reasons why that was a very bad idea, but I think it explains why that temptation existed, because officers do seem to shirk. 

 

Aaron [00:42:19] Now, on the other hand. The good news here, right, is that officers appear to not be very willing to make wrong arrests in order to secure access to overtime pay. They don't seem to be super motivated by money, at least along this margin. And while officers do seem to shirk. It's also true that the arrests that they abate are the lower quality arrests, the arrests that would have had lower conviction rates and lower rates of sentencing. And I think these findings suggest that officers do, in fact, exhibit some pro-social motivation towards their work and that they're not very corruptible, at least at this particular margin. And so overall, I think what this suggests, right, is the idea that police officers aren't very different from other workers, that they balance personal and professional motivations. They care about a mix of money and leisure time, but they care a whole lot about leisure time. And I think that means that caricatures of officers as these profit maximizing machines that look at everyone walks by them as a personal piggy bank aren't accurate, at least in this particular city. You know, I wouldn't want to make claims broadly about the entire world because I do think there are countries where public servants engage in predatory behavior in exchange for bribes and profit. But it's not something that we see looking at policing in Dallas. 

 

Jennifer [00:43:37] Are there any other papers related to this topic that have come out since you and Felipe first started working on this study? 

 

Aaron [00:43:43] Nothing that's directly directly related as far as I've been able to find. This is a pretty new area of research, especially in economics. But there's a couple of working papers that I think are at least in the same family of work that I think are worth mentioning. These are papers that are about how police respond to perceived outside threats. And I think that says something about how police orient towards their work and the degree to which they're pro-socially motivated. So there's a new working paper by Canice Pendergast, who's been a giant in this literature that studies how Los Angeles police officers responded to federal oversight during the late 1990s. So the paper shows that officers responded to more oversight by strategically pulling back and being less proactive in how they police on the streets. There's a similar paper by Tanaya Devi and Roland Fryer, which looks at more recent federal investigations. They find that when a federal investigation is precipitated by what they call a viral incident, so some kind of really bad thing that happened on camera and that the public took notice of, there is a big decline in police productivity, so police pull back, they make fewer stops and then fewer arrests. And then there's this third paper by Bocar Ba and Roman Rivera that shows something similar, but actually provides a little bit more depth in understanding what's happening. So they also find that in the aftermath of a scandal or viral incident, officers pull back. But interestingly, when the police union issues a private memo warning officers to just be more careful, like don't get yourself caught up in a scandal or anything like that. Officers respond by using less force, but they're not actually responding with less ethical policing. 

 

Aaron [00:45:30] And I think an overall takeaway from these papers is that officers are really aware of how much discretion they have on the job, and they're using that discretion accordingly. And I think that's just one more dimension of understanding police motivation. 

 

Jennifer [00:45:44] And what's the research frontier? What are the next big questions in this area that you and others will be thinking about going forward? 

 

Aaron [00:45:51] Yeah. So we are, I think, really starting to crack open the black box of policing, right. There is a lot of ethnographic work that proliferated over the last thirty years. And now, with access to better and better administrative data, researchers are starting to construct quantitative tests of a lot of the theories that have been out there and using data to understand things like, well, how is police officers behavior affected by their peers or the people who train them? How does officer productivity depend on whether they police alone or they policing teams? How do police respond to more managerial oversight? And I think we're just learning more and more about the importance of management and training in policing. 

 

Aaron [00:46:32] And I think to answer your question of the choose to answer this at a very high level, rather than focusing very specifically on financial motivations, I hope that's okay. I think there's three areas that I think really, you know, researchers need to focus on here. So first, there are I think some very well-founded fears that police departments are facing an imminent recruitment crisis. Departments have become more diverse in the last generation. Those cohorts are starting to retire. How can we get qualified people, and in particular, qualified minority candidates to want to be cops? I think there's a real concern that the calculus has changed post George Floyd and and I think finding ways to do that is really important. Second, I think we need to know how to incentivize officers to engage in policing that is effortful, but also economizes on risk to citizens. Police departments have been thinking about this for a long time, but they're more than ever rolling out interventions like procedural justice training and de-escalation training. And we need more evaluation work in this area. There's some - we definitely need more. 

 

Aaron [00:47:36] And then finally, I think we need to expand the amount of research that we are doing on the informal training. So cops in the U.S. get way, way, way less formal training than cops in other  other developed countries. So we don't have a National Police University. Most officers get something like maybe six or seven or eight months of classroom instruction and then they're on the job. A lot of their training occurs informally through mentoring, through learning on the job. Often, you know, cops are told, hey, forget about what you learn in the academy. We'll teach you how it's really done on the street. And so it's not uncommon, for example, for police officers to drive without wearing a seatbelt, even though traffic accidents are usually the leading cause of death for cops. So something that's pointed out in a really interesting ethnographic paper by Michael Sierra-Arevalo, a sociologist at UT Austin, he notes that this is not something cops are taught to do in the academy. It's something that they learn informally. And so what are all of the other informal things that police learn about how to do good police work? You know that they could be learning more of or are they learning things that are not conducive to that, and I think research can really help to sort that out. So those are my big three. 

 

Jennifer [00:48:48] Great. My guest today has been Aaron Chalfin from the University of Pennsylvania. Aaron, thanks so much for talking with me. 

 

Aaron [00:48:54] My pleasure. 

 

Jennifer [00:49:01] You can find links to all the research we discussed today on our website problecausation.com. You can also subscribe to the show there or wherever you get your podcasts to make sure you don't miss a single episode. Big thanks to Emergent Ventures for supporting the show. And thanks also to our Patreon subscribers and other contributors. Probable causation is produced by Doleac Initiatives, a 501(c)3 nonprofit, so all contributions are tax deductible. If you enjoy the podcast, please consider supporting us via Patreon or with a one time donation on our website. Please also consider leaving us a rating and review on Apple Podcasts. This helps others find the show, which we very much appreciate. Our sound engineer is Jon Keur with production assistance from Nefertari Elshiekh. Our music is by Werner, and our logo was designed by Carrie Throckmorton. Thanks for listening, and I'll talk to you in two weeks.