The Federal Reserve Bank of San Francisco published a letter that tries to look at whether or not regulation and taxes are impacting state level employment. (Hat Tip: MFK)
To look at whether or not regulation and taxes are an issue, they use state level data from the National Federation of Independent Businesses (NFIB) monthly small business surveys and correlate it to state level unemployment. The source they linked to didn't seem to contain the state level data they were given, so unfortunately I am confined to critiquing their economic letter rather than improving upon it.
If you want to look at the only interesting part of their paper, look at this following chart:
It turns out that places with large amounts of household debt relative to income in 2006 ran into problems with sales during the economic crisis. By backing up their previous research with the small business survey, the authors try to show the usefulness of the NFIB survey and imply that because they can't find the same data relationship between business leaders' changing worries about regulation and unemployment that there must not be a significant impact from those regulations. However, their attempt runs into a few problems.
1. The study looks at percent change in unemployment in many cases - I'm sure North Dakota which has 3.2% unemployment (but it went up by over 10% in their scatterplot!) isn't actually the worst off as it implies in this chart.
2. They are looking at the change in the percentage of businesses who are worried about regulation and taxes as their top concern. Again, this is a percent change in a percent - so a change from 10% to 15% of businesses worrying about regulation and taxes is more significant than a shift from 25% to 35%. They are also looking at the change from 2008 through 2011, which largely overlaps with the rise of the tea party. A survey question about sales being a problem isn't that political, but a survey question about taxes and regulation will be far more correlated to political shifts than the actual impact of those regulations.
If Amir Sufi and Atif Mian wanted to do an analysis regarding the impact of high taxes and regulation uncertainty that might look it was done by professors from Princeton University and the University of Chicago Booth School of Business how could they go about it?
First, they should control for potentially exogenous variables like the political shift that came with the rise of the tea party. There is a lot of poll data out there, so seeing which business leaders were complaining about regulation, adjusting for the change in political views of their state, could contain a lot more information. They should also look to use poll data that ask specifically about state regulations, which is more likely to have a direct connection to state level unemployment. Without adjusting the data like this, it leads to silly inputs that show both California and Texans dealing with similar tax and regulation problems.
Second, using percent changes of percentages can be very noisy. At the very least, differences in unemployment as opposed to percent changes in unemployment should be looked at alongside the absolute levels (which they did look at).
Without at least these two steps, a single or multivariate regression wouldn't show any effect from regulations even if there was one. But on top of that, it should be noted that unemployment statistics don't tell the whole story. In Texas it's entirely possible that lower regulation did drive job growth, but the population growth that occurred alongside the job growth has kept the unemployment rate high and therefore this data wouldn't show up in their analysis.
There are lots of ways that this analysis could have been very interesting. It's too bad the authors didn't try them. At least they did provide another example of academics putting out bad arguments in an attempt to elevate the status of their previously published work.