Rossi's Revenge

Why guaranteed income programs fail; or, the Neoconservative critique

I am contractually obligated as a discourse participator to acknowledge the launch of yet another discourse participating Substack.

The Argument

is an Abundance liberal absolutely definitely not Abundance liberal publication, led by Jerusalem Demsas (ex-The Atlantic) and staffed by a variety of people who absolutely do not have a certain ideological movement in common.¹

The opening substantive post, from staff writer

Kelsey Piper

(whose departure from Vox means I no longer have someone to identify as the reasonable person at Vox), concerns the recent, consistent failure of guaranteed income pilots to produce any non-pecuniary benefits for their treated participants. This finding is, as Piper puts it, “shocking”:

Homeless people, new mothers and low-income Americans all over the country received thousands of dollars. And it's practically invisible in the data. On so many important metrics, these people are statistically indistinguishable from those who did not receive this aid.

I cannot stress how shocking I find this and I want to be clear that this is not “we got some weak counterevidence.” These are careful, well-conducted studies. They are large enough to rule out even small positive effects and they are all very similar. This is an amount of evidence that in almost any other context we’d consider definitive.

Piper is nonetheless undeterred in her optimism about more targeted transfers, and about social policy more generally. This optimism—the more precise term might be meliorism—is characteristic of the liberalism that Piper, Demsas, et al. claim they would like to revive.

Yet it is hardly unprecedented for that optimism to run aground on the realities of policy evaluation. Indeed, liberal meliorism has faced precisely the same challenge before, decades ago. And the same basic explanations for why still obtain.²

Many people who have problems—emotional, psychological, interpersonal, etc.—also do not have money. One reasonable inference is that the cause of those problems is the lack of money. A solution to many of our social problems, therefore, might be to do as Annie Lowrey once enjoined, and “give people money.”

But what actually happens when you give people money? A series of increasingly large, increasingly well-funded randomized controlled trials have in recent years attempted to answer this question. These target slightly different populations, but they all work essentially the same way: recruit a large random sample; split them into treatment and control; give the treatment group a large amount of money per month (between $333 and $1,000) and give the control no or a token amount of money per month; see how the two groups compare.

What do these studies show? As Piper summarizes it, “guaranteed income transfers do not appear to produce sustained improvements in mental health, stress levels, physical health, child development outcomes or employment. Treated participants do work a little less, but shockingly, this doesn’t correspond with either lower stress levels or higher overall reported life satisfaction.”

Just to elaborate: the primary effect of transfers is that recipients have more money than non-recipients. Secondarily, recipients work slightly less, because they replace some but not all of the lost income with program dollars. But … that’s about it. Neither participants’ physical nor mental health improves. They don’t get more education or build skills. Their broader financial position doesn’t improve. Their parenting doesn’t improve. Their political participation doesn’t improve.

Why is this? Piper suggests that perhaps the improvements just aren’t showing up in the measurements:

Maybe this isn’t as confusing as it initially feels. If you gave me an extra 10% of my income to spend, it would absolutely make me better off — just not necessarily in ways you could measure.

I might give some to friends going through a time of need, which makes my community stronger even if it doesn’t show up in a study looking at my habits. I might take the kids across the country to visit my aging grandparents — stressful, but worth it in a way that probably doesn’t show up on a mental health inventory.

This is, and I don’t know how else to put it, a silly argument. Null effects show up across more or less every non-pecuniary measure across multiple RCTs. Blaming measurement error is unfalsifiable. Your response shouldn’t be “no randomized controlled trial is really going to convince me that money does not improve people's lives.” When the evidence so overwhelmingly says that money doesn’t change non-pecuniary outcomes, you need to change your assumptions.

And this is particularly so because big, bold nulls are not actually that unusual in the world of policy evaluation! In my own primary area of focus, criminal justice policy, transfers sometimes reduce monetary crime (mostly either by substitution or by getting people off of drugs), but basically don’t affect violent crime. Educational interventions? Often nulls. Or take something as simple as giving people health insurance: the famous Oregon Medicaid experiment found “no significant improvements in measured physical health outcomes” in the treatment versus control groups.

The prevalence of nulls in evaluations of social policy interventions is so well-established that there’s actually a term for it. It’s called Rossi’s Iron Law, after sociologist and policy evaluator Peter Rossi. Rossi’s iron law is: “the expected value of any net impact assessment of any large scale social program is zero.” There are a handful of corollaries, like the stainless steel law (“the better designed the impact assessment of a social program, the more likely is the resulting estimate of net impact to be zero”) and the brass law of evaluation (“The more social programs are designed to change individuals, the more likely the net impact of the program will be zero”).

Rossi is a funny figure, actually. In addition to his work in policy evaluation, he was an ardent liberal of the Piper variety. His expertise in evaluation came partly out of his work on War on Poverty programs, part of the broader Johnson-era, Great Society push for government to do more to ameliorate the condition of the poorest Americans. His association with policy pessimism, in other words, is in spite of his politics and his vocation.

Yet Rossi’s formal pessimism was not atypical for the liberal establishment of his era. The Great Society was meant to work the kind of transformation that UBI advocates sometimes allude to. The goal, importantly, was not just to give people money, but to help people—through transfers and also government programming—overcome the misery and dysfunction which alarmed leaders like Johnson. They wanted to make “taxpayers out of taxeaters,” in an evocative Johnsonian turn of phrase—to help citizens become independent through government support.

At the macro level, more or less the opposite obtained. Beginning in the late 1960s, social pathology exploded: crime, drug use, family breakdown, teenage pregnancy, urban collapse, and a general personal and social apocalypse. Concurrently, many social scientists observed that many social policy interventions did not obtain their desired goals. Charles Murray, the social scientist and prominent public controversialist, made his name arguing that the Great Society had caused all of these dysfunctions, by subsidizing the lifestyle of the lower classes. On the back of this theory was borne welfare reform, among other initiatives.

Personally, I am less partial to the Murray theory and more to one that might be identified with Rossi, but more likely with the public intellectual Senator Daniel Patrick Moynihan. In the report that launched him to infamy, Moynihan—then a deputy at the Johnson labor department—argued that the problems then faced by black America were the result of a “tangle of pathology,” itself downstream of a deep sickness of family structure with roots in slavery. Moynihan’s slavery argument was probably wrong (William Julius Wilson, another noted progressive sociologist, demonstrated this later). This is especially so because the tangle of pathology has grown to encompass much of white America, a phenomenon documented both by Murray and the Vice President.

Which is not to say that guaranteed income experiments fail because of family structure per se. Rather, the point is that the implicit causal model of these experiments (less money → more problems) is almost certainly wrong. One or more additional, hard-to-alter variables is or are likely causing both people’s lack of money and their other problems, such that addressing one does not address the others.

What is that variable? There are lots of explanations. If you’re Murray you talk about biology; if you’re Moynihan, you talk about family structure; if you’re William Julius Wilson, you talk about deindustrialization. Heck, there are even left-wing variants here: “critical race theory,” before it was an object of cultural discourse, referred to the left-wing legal tradition that was argued that the civil rights revolution had not ameliorated the black condition in America, which was taken as foundational evidence that deeper, more “structural” racism may be at play.

Which of these explanations is correct? Well, that’s … a causal fallacy. I tend to prefer a simpler formulation: for whatever reason, it is very, very hard to change people using policy. (This is Rossi’s brass law.) If you give people money, their bank accounts will be a little more full, and they might work a little less to compensate. But they won’t become healthier, or better educated, or more responsible with money, or better parents, or better citizens. They’ll be the same people with a little more cash.

This doesn’t mean that policy can’t do things. It can, and often does. But as a general rule, when it does it does so through simple channels that involve shifting people’s incentives, rather than trying to change people as people. And because those channels are few and far between, we should expect policy that tries to do the hard work of changing people to mostly fail.

There’s actually a term for this line of critique, as applied not merely as an evaluative insight but a political one. That term is “neoconservatism.” Long before it took on today’s pejorative sense,³ it referred to a group of previously left-of-center intellectuals who were persuaded, largely by their encounters with actually existing government policy as implemented under the Great Society, that policy rarely worked, often backfired, and reliably could not change who people are.

This original group of neoconservatives—Moynihan, James Q. Wilson, Irving Kristol, Ed Banfield, and so on—more or less decisively won the policy debate in the 1980s and 1990s, such that liberals like Rossi (and William Julius Wilson, and James Coleman, and Christopher Jencks) were often forced to grant their premises. Indeed, part of how we ended up at the idea of unconditional cash transfers is that giving people money became something of an alternative to the more comprehensive social services the Great Society had once envisioned, but which fell out of fashion when it seemed like they weren’t working.

All of this intellectual history brings me back, in a roundabout way, to Piper and her shock at the failure of guaranteed income. If you are, as a matter of principle, committed to the idea that giving people money will solve their problems, then I cannot help you. But if you are willing to see the evidence that Rossi was right, and that social policy can rarely change people, then the failure of UBI should be for you what Irving Kristol famously referred to as the experience of being “mugged by reality.”

You can, that is, insist that the evidence has failed your principles. But in my experience, if the evidence is against your principles, it’s not the evidence that may need to change.

1. Not that there’s anything wrong with that!

2. This piece is distinct from, but retreads in a different order many of the same ideas in, a piece I wrote for Fusion about a year ago. If you would like supplementary reading, consider it.

3. Roughly: “guys who favored invading Iraq;” or, in certain corners of the internet, “Jews who favored invading Iraq.”