Bad Statistics, Worse Economics: Refuting a New Minimum Wage Study

The National Employment Law Project, a labor advocacy group, just released a new study of the minimum wage, which aims to discredit the idea that raising the minimum wage kills jobs. The study looks at every federal minimum wage increase ever enacted (22 in total) and producing some eye-catching statistics. Here are a few of the most important:

  • In 68 percent of cases, overall employment increased after the minimum wage was increased
  • In 82 percent of cases, employment in the Leisure and Hospitality sector increased after the minimum wage was raised
  • In 73 percent of cases, employment in the Retail sector increased after the minimum wage was raised

These are interesting results because they appear to run counter to economic theory. How do we account for this?

The short answer is that it’s pretty easy to make statistics say whatever you want them to. And this is particularly true in economics, where true experiments are impossible and nothing ever happens in a vacuum. The analysis conducted by the authors is technically correct from a statistical point of view, but it does not mean what they claim it does.

The Importance of Ceteris Paribus

The problem is that the authors forgot about that all-important phrase in economics, ceteris paribus. It means “all things equal” and you’ll see it, or its translation, appended to just about every economic model you ever encounter. It’s everywhere, and it’s essential. The economy involves an almost unlimited number of variables that affect the behavior of individuals in one way or another. But most models only consider the relationship between a very small fraction of items, like prices, wages, and quantity. Thus, for any of these models to mean anything at all, they always come with the consummate disclaimer: ceteris paribus–all things being equal.

This condition is necessary to prevent economists from looking foolish more often than they already do. Let’s imagine a local shoe cobbler asks an economist how many pairs of shoes he will sell if he lowers the price from $2 to $1 a pair. The economist, with the model of supply and demand in the back of his mind, astutely says that the cobbler will sell more shoes, all things equal. Then a tsunami comes and destroys all of the townspeople’s possessions, but curiously, spares the shoe cobbler for the sake of our example. Now despite lowering the price, the cobbler does not sell more shoes. The economist explains that he wasn’t wrong; his advice was only correct if other factors didn’t change. The tsunami was a big change, so the economist is vindicated. And the shoe cobbler learns the important lesson of not asking an economist for business advice.

Back to the Study

So how does any of this relate to the minimum wage study? Well, the authors in this case are making a very simple calculation for each of their headline statistics. They take total employment–that is, total bodies working–in period 1 right before the minimum wage increase occurred. Then they compared it to total employment (again, people working) as of exactly 12 months later. They also run this same analysis looking at total employment in specific sectors thought to be most affected by minimum wage changes. And again, the measure of employment being used is just the number of people working.

If employment increased over the 12 month period, then there is a positive correlation, and they interpret that as showing that the minimum wage had no negative consequences. If employment decreased, then, in their analysis, this would represent a case where the minimum wage may have had a negative effect. Of course, I say “may” here because the authors actually explain away all the periods with employment declines as follows:

…the small minority of instances in which employment—either overall or in the indicator sectors—declined following a federal minimum-wage increase all occurred during periods of recession or near recession. That pattern strongly suggests that the few instances of such declines in employment are better explained by the overall national business cycle than by the minimum wage.

Hold on. So in the negative cases, we can’t really say anything about the minimum wage because the broader economic trend was downward. But in the positive cases, where the general economic trend was upward, that counts as “evidence” that the minimum wage doesn’t cause any harm. See what’s going on here? Heads, they win; tails, their opponents lose.

The reality is that looking at the change in overall or total sector employment, across the entire country, before and after the minimum wage hike, cannot tell us anything about the minimum wage’s effects. There are too many variables, and this approach doesn’t attempt to control for any of them. Instead of all things equal; it’s all things variable.

Here are just a few of the factors that are likely to have a dramatic positive effect on employment (total and sector-specific) that have nothing to do with the federal minimum wage:

  • Immigration to the United States
  • The number of young people who are newly old enough to enter the workforce
  • The entry of women into the workforce

Add to this the fact that the number of people that actually make the federal minimum wage is much smaller than people imagine. When statistics on this first started being captured in 1979, just 13.4% of hourly wage workers made the minimum wage. By 2014, that figure had dropped just 3.9% of hourly wage workers, roughly 3 million people, qualified for the minimum wage. The figures were probably higher earlier in the century, but what this shows is that the impact of the minimum wage on total national employment is likely to be drowned out by other factors in any case. This doesn’t mean the minimum wage isn’t harmful. It just means that its effect (positive or negative) isn’t likely to be captured in nationwide aggregate employment statistics because it only affects a small portion of the workforce.

Thus, in reality, this study is not rebutting the usual, measured arguments against the minimum wage. It is rebutting one very specific and outlandish version of the argument, which must run something like this: “Raising the federal minimum wage by any amount will lead to so many job losses that it will overwhelm and wipe out any and all job growth that would be produced by new immigration, young people entering the workforce, and women entering the workforce.” Of course, the problem is that no sensible economist has ever made such a claim. So refuting it is not helpful–except for the present purpose of explaining how not to do economic analysis.

An Alternate Analysis

Incidentally, because I was curious and the authors did commendably publish their data, I decided to run my own quick analysis that would account for the important all things equal condition. Given all the complications outlined above and many others, this still shouldn’t be taken as anything close to good evidence. But it is useful for showing how dramatically the results can be swung when using a distorted methodology. Since I’m lazy, I had no desire whatsoever to track down rates of immigration or labor force growth for the purposes of my back-of-the-napkin run. Instead, I did a comparative analysis.

For this purpose, I assumed that the Leisure and Hospitality (LH) sector and the Retail sector would both be hit harder by a minimum wage increase than the overall economy. This makes sense because the proportion of minimum wage jobs is likely to be higher for these sectors than it will be for the overall economy. Thus, I calculated the annual growth rate in total employment over the 12 month period after a minimum wage increase and compared it to the employment growth rate of the LH and Retail sectors combined. If the overall economy’s employment grew at a higher (or less negative) rate than LH and Retail sector employment, this would suggest the minimum wage may have had a negative effect. If total employment grew faster in the LH and Retail sector, then this would imply the minimum wage did not have a negative effect (or may even have had a positive effect). By comparing relative growth rates over the same economic period in this way, I am holding constant some of the broader economic trends that eluded the authors’ methodology.

So what were the results? I found that 45% of federal minimum wage increases were associated with stronger performance in the LH and Retail industries as opposed to the overall economy. That is, 45% of minimum wage increases were associated with a positive impact, in contrast with the study’s original claim that 68% of wage increases were associated with positive employment changes. But again, I’m not about to claim this proves my case because of all the limitations discussed above. Economic theory itself is far more compelling in that regard, as we have previously explained.

I am certainly belaboring the point now, but I think one more item is worth noting. For sport, I also decided to look at the total change in employment across every 12-month period in the data, which ranged from 1939 to mid-2015. Here, I found that 77% of all 12-month periods were associated with increased employment. And this should make sense; this was a period of immense economic and population growth for our country. Rising employment should be the rule, not the exception. But look what this also means. If we just randomly selected a 12-month period and compared ending employment to initial employment, there is a 77% probability we’d find an increase in employment. However, when the authors selected only those periods associated with a minimum wage increase, only 68% showed positive employment change. Of course, that’s not proof that raising the minimum wage will ruin the world. But it’s also not a ringing endorsement for the economic soundness of a minimum wage as the authors would like to pretend.

Summing Up

In closing, I should be fair to the authors and note that neither of them claim to be economists. The results they found could have been the result of shrewdness or ignorance, and for our purposes, it doesn’t really matter which is true. But whatever their intent, the net result of their analysis is the same as the one commonly deployed to critique economic theory. They created a straw man by misrepresenting the other side’s views (that minimum wage increases would cause absolute losses in employment) and then they weakly disproved the myth they created. In this way, it’s another illustration of why we should endeavor to understand our opponents’ ideas before we attempt to criticize them.

In the authors’ own summary, they suggest that opposition to minimum wage is “rooted in ideology, not evidence.” But if this is what counts as evidence these days, I’d recommend sticking with ideology. You can be the judge.

Leave a Reply

Your email address will not be published. Required fields are marked *