1 Comment
Sep 30, 2023Liked by Charles Fain Lehman

A few different points:

IMO state macro level synth designs that use placebo tests are often underpowered. This is both because of limited placebo pool, as well as the default synth method can be quite volatile. (Prefer a regression approach that is not as common, as well as the conformal inference method you mention in the footnote, https://andrewpwheeler.com/2019/12/06/using-regularization-to-generate-synthetic-controls-and-conformal-prediction-for-significance-tests/.)

For the 2018 part, it is likely the authors will argue (somewhat reasonably) that there has been a regime change in the nature of fentanyl around that time. Now, that offhand does not explicitly make it so the counterfactual co-variation trends are different though (the 1999 example you use would be harder to argue with, regime change due to way data is estimated).

Probably a stronger argument would be the nature of the drugs being traded in west coast vs east coast (which applies over the entire period? I feel like that is maybe an idea I got from something you wrote Charles). But that might invalidate the whole design. (Again though using fuzzy arguments.)

Other tidbit, default synth doesn't do so well with more volatile monthly data. (Fit would be horrible if going back to 1999 using monthly.) Some might argue your pre-fit is too smooth in just those few years for example (although from a bias-variance standpoint, I think lower variance is a good thing).

Expand full comment