The setting is a 'mover' event study, where individuals move across cities; the treatment is the change in city environment. In M21, this is the change in cluster size, defined as the number of inventors in the same research field and city. The treatment variable is Size_diff = Size_post - Size_pre. The event study interacts Size_diff with event time indicators.

The sample is movers: inventors who change cities exactly once. Stayers are excluded from the sample (i.e., there are no never-treated observations). It's a staggered rollout design, with inventors moving in different calendar years. The graph above includes individual, year, and city fixed effects. Seems fine, right?

When I simulate data for N=1000 individuals moving between C=20 cities, with true effect B=1, we get the expected event study graph (using the same fixed effects):

Here's the graph for N=1000 individuals and C=1000 cities. This looks very similar to the original graph! So the city fixed effects are the key.

The core issue is cities being observed only as origins or only as destinations. By definition, origin = (Post==0), so origin-only cities have Post=0 for all observations. Similarly, destination-only always have Post=1. For these observations, the city fixed effect is collinear with Post.

In the event study, with t=-1 omitted, this means the post-move indicators are collinear with the city FE. In the extreme case, where origins and destinations are disjoint, no cities are both origin and destination, so the regression is unidentified.

With C=1000 cities, only 5% of city-year pairs are observed as both origin and destination. So when including city fixed effects, the effective sample is very small, and we get a big jump in confidence interval width. With C=20 cities, this is 78%, so the confidence intervals are fine.

Note that the jump in confidence interval width is just a function of the omitted period. When I omit t=0 instead of t=-1, then the pre-move indicators are nearly collinear with the city fixed effect, and the confidence intervals are now wider pre-move.

Final note: the pattern of large standard errors could also be explained by treatment effect heterogeneity. In this case, however, it is due to the city fixed effects driving collinearity. When I redo the original graph using real data and remove the city fixed effects, the jump in width disappears.

Link to simulation code.

But why are there two panels? An event study gives us only one set of coefficients, i.e., the treatment effect in each event year. What does the spike in Figure 6a mean? Why isn’t t=-1 omitted as the reference year? What’s going on with the x-axis labels?

To explain this, let’s go back to Figure 5 (which happens to look very similar to Figure 6). M21 is running a regression of log patents (y) for inventor i on current log cluster size (S) as well as five leads and five lags of cluster size. Here’s M21 Equation 2:

The lead terms (B_1 to B_5) capture how current patenting is affected by future cluster size, while the lag terms (B_{-5} to B_{-1}) capture lagged effects of past cluster size. M21 says this is “studying the dynamic response of productivity following a change in cluster size”, but also that “this is not a standard event study”. (p.3351)

Thanks to Schmidheiny and Siegloch (2023), we know that this is a distributed lag model. They prove that event study (ES) and distributed lag (DL) models are equivalent.1 But ES models give you estimates of the cumulative effect of a treatment, and DL models give you the incremental effect, i.e., changes in the cumulative effect. So the DL estimates need to be aggregated to get the total treatment effect. In M21 Fig 5, the beta terms are the incremental effects and the mu terms are the aggregated cumulative effect.2

Moreover, this is the most extreme case of a DL model. Cluster size is a continuous variable and changes every year, so this is equivalent to an event study where a different event occurs every year (i.e., a change in cluster size). The simplest case is a binary treatment that turns on once; here we have a time-varying continuous treatment dose. M21 does not discuss the identifying assumptions required here.

Now let’s revisit the event study in Figure 6. M21 seems to think that this is also a DL model, with incremental beta estimates in the top panel and aggregate mu estimates in the bottom panel. The spike in Figure 6a is exactly what a proper treatment effect looks like in a DL model, where there is an incremental effect in the treatment year, and nothing in other years. But this is wrong: an event study already gives us aggregate treatment effects, so aggregating again means calculating the cumulative response function of a cumulative effect. So Figure 6b is meaningless and should be removed.

And the text clearly states that Figure 6 is attempting an event study, based on interacting a treatment variable with event time indicators:

In Figure 6 I replicate [Figure 5] using only variation from inventors who move across cities. [...] I use the subset of inventors who change city once to estimate a variant of equation (2), where t = 0 marks the time of the move, and the timing relative to an inventor’s move is interacted with the corresponding cluster size before the move and after the move. [...] Thus, this specification is an event study, based on a “pure” movers design solely exploiting variation in cluster size induced by moves.” (p.3352)

But as I discuss in my comment, this is not a proper event study specification. Instead of using the change in cluster size associated with moving across cities, M21 interacts pre-move average cluster size with the pre-move event time indicators, and post-move average cluster size with the post-move event time indicators. M21 doesn’t show a regression equation; my reconstruction is:

But this doesn’t use variation in cluster size from moving; it is identified even if there is no change in size (Size_pre = Size_post). In my comment, I show that a correct mover event study using ∆Size = Size_post - Size_pre as the treatment variable gives a null result.

So why does Figure 6a have that spike in t=0? Equation 1 is not using variation from moving, so there shouldn’t be a spike. This specification is actually testing for heterogeneity in the cross-sectional relationship between (averaged) cluster size and patenting by event time. So the beta coefficients should all be equal, if they estimate the same constant elasticity.

The problem is that the M21 code does not follow my Equation 1 above, due to a coding error. It doesn’t generate an interaction of Size_post with the event time indicator for t=0, but instead uses the regular time-varying cluster size variable (ln S_it). So B_0 is estimated using data from all event years, instead of capturing the effect in t=0. In Figure A1 of my comment, I show the results using the original and corrected code. The large spike disappears:

To sum up: M21 Figure 6 shows an attempted event study that is confused with a distributed lag model. Figure 6b calculates the cumulative response function of a cumulative effect, and is unnecessary. Figure 6a uses an incorrect event study specification, and the large effect is due to a coding error anyway.

M21 studies agglomeration effects for innovation: do bigger technology clusters have more patenting? The main results are based on inventor-level regressions of patents on cluster size, which give a positive correlation. To claim a causal effect, M21 uses an event study and IV strategy to rule out selection and omitted variable bias.

To test for selection bias, M21 uses an event study based on inventors moving across cities. Since moving changes cluster size, we should see a clear effect on patenting; if inventors select into bigger cities, we’ll see a trend before the move. I show that the original event study uses an incorrect model: it doesn’t actually use variation from moving. When I run a proper mover event study, I get a null result.

M21 uses an instrumental variable to address omitted variable bias, such as local subsidies that drive both cluster size and patenting. The instrument is based on the number of inventors in the same field who work for firms in other cities. Due to a coding error, the data is not sorted by city, so the instrument is constructed incorrectly. I correct the code and find a null result.

The event study and IV are necessary to establish that the baseline regressions show causation, nor correlation. Since they both fail, the main takeaway of my comment is that the positive correlation between cluster size and patenting may not be causal.

I was hired by Coefficient Giving to do a small extension of this paper, since it’s the main evidence on technology agglomeration effects that informs their housing policy funding. But the more I looked at the paper, the more problems I found. Let’s go through the other eight problems:

Issue 3: Early on, I would sometimes get different results when running the baseline regression to get the main patenting-size elasticity. It turns out this is due to M21 using many-to-many merges in the data cleaning, so the results are actually unreproducible. Each time you run the code, you get a slightly different sample.

Issue 4: Table 6 studies the effect of cluster size on patent quality, measured using citations. M21 claims to use log citations, but the code actually does log(y+0.00001). When I use log(y+1) or Poisson, the effect switches from positive to negative.

As in Chen and Roth, this happens because the extensive margin effect is positive, and a smaller constant puts less weight on the intensive margin effect. Here, c=0.00001 means valuing the extensive margin as a 40,000% change in the intensive margin.

M21 also calculates citation- per-patent using citations-per-coauthor and patents-per-coauthor; but this nullifies the per-coauthor adjustment. The correct approach is to use citations per coauthor / whole patents.

So the original M21 finding is reversed: patent quality is in fact decreasing in cluster size. The actual story may be: larger clusters produce more patents, but they are lower quality, e.g., strategic patents used in IP lawsuits. (For example, bigger firms produce strategic patents and locate in big cities.)

Issue 5: there’s another substantive coding error when M21 estimates heterogeneous elasticities by cluster size in Table 8. Here the goal is to test whether agglomeration effects are bigger in bigger clusters. M21 reports limited heterogeneity.

M21 interacts log cluster size with size quartile indicators, but incorrectly omits the quartile indicators themselves. This forces the baseline level of patents to be equal across quartiles, biasing the estimates. When I do the correct interaction, I find substantial heterogeneity across size quartiles. The patenting-cluster size elasticity is actually much bigger in large clusters.

M21 does an exercise to quantify the aggregate benefits of agglomeration. The idea: equalize cluster size and calculate the decrease in the total number of patents. Since inventors are reallocated to smaller clusters, total output falls. But since M21 understates the heterogeneity in elasticities by size quartiles, M21 also understates the aggregate effects. The aggregate losses appear to be 3x larger!

But this isn’t the final word. First, the elasticity may not be causal. Second, we need to account for the negative effect on patent quality. When I redo the exercise using quality-adjusted patents, I find a similar aggregate loss as the original. (See appendix fn.32)

Issue 6: M21 uses a distributed lag model in Figure 5, regressing log patents on five leads and five lags of log cluster size. This is a demanding model, and M21 doesn’t discuss identifying assumptions. (More discussion here.)

There’s another coding error: M21 defines leads and lags using Stata’s _n operator, which refers to the observation number. But with an unbalanced panel, the previous observation is not the previous year. (You should use the panel operators L. and F.) When I fix the code, the estimate loses significance.

This also applies to Fig A1, which restricts to the subset of pharmaceutical patents. When I correct the code, the sample size drops to N=58, so the regression fails entirely. Moreover, the M21 code doesn’t even match the published figure: there’s no B_0 term in Fig A1, but it’s there when I run the original code. But Fig A1 is based on Equation 2, which clearly has B_0.

Issue 7: M21 Table A.8 tests whether the elasticity is larger in higher-quality clusters. There are two issues here: the code is supposed to use modal cluster size, but M21 now takes average size. And M21 doesn’t adjust for team size, when the main results use fractional attribution for coauthored patents. After correcting, the estimates are half as large.

Issue 8: M21 varies the time unit of the data, from 1-month to 3-year periods. The idea, according to M21, is to address missing observations; when inventors do not patent, they don’t have a row in the data. M21 claims this creates a downward bias at shorter time units, because more of the extensive margin effect is missing.

This interpretation is incorrect. Regardless of the time unit, there are no observations with patent=0, so no extensive margin effect can be estimated. Moreover: at shorter time units, coauthors make up a bigger share of cluster size, and patents-per-coauthor is mechanically linked to the number of coauthors. This creates a mechanical negative bias.

Issue 9: M21 claims that controlling for team size (number of coauthors on a patent) increases the elasticity. This is incorrect: patents are already measured on a per-coauthor basis, so adjusting for team size again means controlling for the same variable twice.

This is especially confused. Team size is not a confounder, but a mediator: cluster size affects patenting directly, and indirectly by enabling larger teams, where team size independently affects patenting. Controlling for a mediator means shutting down the causal pathway we’re trying to estimate.

Issue 10: When inventors do not patent in a year, they don’t have a row in the data; we know they have zero patents, but their city is unobserved. M21 interpolates these observations for gaps of 1 or 2 years, to capture part of the extensive margin effect.

There’s a coding error: for two-year gaps, only the second year of the gap is filled. Fixing this increases the elasticity, since more observations are interpolated, leading to a bigger extensive margin.

So there are a lot of issues in M21. The major ones are the event study and IV, the citation results, and the heterogeneity by cluster size. Should you believe in agglomeration effects for innovation? My advice: don’t update too much on this paper. (Although I do have ideas on how to fix it.)

I started working on this 3 years ago. I emailed Moretti about the event study in July 2023, and never got a response. But he was a referee at AER and offered some useful feedback. It would have been helpful for Coefficient Giving to get this feedback sooner!

This has happened to me several times now, where the author never responds to questions. If you publish research, and it’s actually being used to inform decision-making, please respond to replicators! (Another case was my replication of Atwood 2022 on the long-run economic effects of the measles vaccine.)

Again, read my full comment here. Please support my work by becoming a paid subscriber.

I. An incoherent theory of land values

Condon’s thesis is that rising land values are responsible for increasing housing costs:

The thesis of this volume […] is that the housing crisis is not caused by an impeded supply of homes but by the asset value of the land below them. This Rent value is inflated when higher-density homes are authorized, much to the disappointment of those who argue that the housing-affordability problem is caused by constrained supply. (p.177)

In other words, since land is an input into the production of housing, higher land values will drive up housing costs. And upzoning in particular is a primary cause of higher land values. This view is superficially plausible. As a naive empiricist, Condon has observed rising housing prices and land values coincide with upzonings. But what is the actual causal relationship underlying these correlations?

Condon thinks that upzoning drives land values, which in turn causes higher housing prices. Right away we can find a fatal flaw in this view: in First Shaughnessy, a heritage-protected mansion neighborhood in Vancouver, land values quintupled between 2006 and 2017 (before decreasing after a property tax was implemented). Over the same period, building values doubled.1

The entire neighborhood of First Shaughnessy is a heritage conservation area, so we have a clean natural experiment: very few upzonings are happening here (and any rezoned parcels are removed from the data), and yet land values are soaring. Moreover, the pattern in the rest of Vancouver is almost identical; so land values are increasing whether or not upzonings occur.2

But then what is causing the increase in land values? Condon has no answer. He could appeal to speculators bidding up parcels in anticipation of future upzonings, but when the entire neighborhood is heritage-protected, this is obviously strained. Hence, we can discard his thesis.

So what is the correct model of housing and land values?

Consider a simplified setting where land is zoned for either detached houses or six-storey apartments; developers can build houses on house-zoned or apartment-zoned land, and can build apartments only on apartment-zoned land. Upzoning means reallocating a parcel from house-zoning to apartment-zoning. This makes apartment-zoned land more abundant and hence cheaper (Figure 1b), while house-zoned land becomes scarcer and more expensive (Figure 1a). Since land is an input into housing, upzoning makes apartments cheaper and houses more expensive.3 (See here for a full writeup.)

A supply and demand model of upzoning and land prices

Condon does get one thing right: the upzoned parcel itself increases in value as it switches markets (from P1H to P2A). But this ‘own-parcel effect’ is irrelevant for evaluating upzoning. Developers make decisions based on the market-wide price, not the price of a single parcel. Upzoning reduces costs for apartment-developers, because it reduces the price of apartment-zoned land. The upzoned parcel itself was previously legally unavailable to apartment-developers, so making it available at some price is an unambiguous improvement.

In this model, housing demand is the main driver of land values. When demand to live in the city increases while supply is constrained, housing prices go up, which raises developers’ willingness-to-pay for land. This is why land values in First Shaughnessy quintupled: Vancouver became more attractive to global-elite CEO types who want to live in a mansion. Hence, the cause of the housing crisis is housing demand growing faster than supply, with rising land values being a symptom. So Condon has misdiagnosed the problem, confusing cause with effect.

We can also see the problem with Condon’s thesis by applying it more generally. His view implies that it’s never worth it to upgrade something, because upgrading just increases the cost. For example, processing iron ore into steel just means you have to pay more for the finished product. Let’s apply Condon’s thesis from above to steel:

the [steel] crisis is not caused by an impeded supply of [steel] but by the asset value of [iron ore]. This Rent value is inflated when [ore-processing is] authorized, much to the disappointment of those who argue that the [steel]-affordability problem is caused by constrained supply.

But of course, manufacturers can’t use iron ore, just as apartment-developers can’t use house-zoned land. And the increased availability of steel makes it cheaper, so manufacturers are actually better off, just as apartment-developers are better off when apartment-zoned land is cheaper.

II. A dogmatic rejection of supply and demand

In Condon’s worldview, supply and demand cannot explain the housing market:

The theory of supply and demand has been undercut by the observed economic reality […] that no matter how many new housing units a metropolitan area adds, housing prices continue to rise. (p.56)

Condon attributes the problem to land values:

[The] increase in urban land price seems to be caused by factors other than the supply and demand for new homes. The increase has to do with the innate limits on the availability of urban land and with how these limits make land perform uniquely in the global marketplace for “real” assets. (p.56)

For Condon, housing demand doesn’t affect land prices, so he needs to find some other factor to explain why land values have increased. As we’ve seen, his attempt to pin the problem on upzoning fails, and with it, his attempt to disprove supply and demand.

In fact, a supply and demand model can neatly explain the data. Demand to live in cities like Vancouver has increased, and restrictive zoning has prevented supply from keeping up. This supply and demand mismatch causes both higher housing prices and higher land values.

Condon is fond of claiming that Vancouver’s city center has tripled the stock of homes, yet prices remain very high. He is simply unable to conceive of demand being a cause of housing prices.

III. A reliance on theoretical edge cases

To see how Condon uses extreme theoretical edge cases, let’s consider his numerical example showing that upzoning merely increases land values:

Imagine a parcel of 4,000 square feet with an allowable floor surface ratio of one (FSR 1) that sells for $2 million dollars prior to rezoning. If the allowable density is doubled to a floor surface ratio of two (FSR 2), the redevelopment value increases in kind, forcing a near doubling in the value of the land to $4 million. Why?

When the city authorities a doubling of market density without requiring affordability, the residual land price goes up in response. The city gets more efficient use of the land with the new density – all good – but gets no substantial decrease in the [price] per square foot of new housing. […]

In this example, with a market sale price of $1,000 per square foot of usable interior space, the developer would calculate a land-price residual of $500 per buildable square foot because that is what is left after paying for construction (at $250 per square foot) and for fees, constants, and profits (at a total of $250 per square foot). The developer can afford only up to $500 per buildable square foot, or roughly $4 million for land (i.e., 8,000 interior square feet at $500 per buildable square foot). […] But note the land-price residual has doubled from $2 million to $4 million. And if the developer refuses to pay the $4 million, some other developer certainly will. (p.195-96)4

Note the last sentence: this entire argument depends on this tacked-on clause, though Condon does not realize it. When would a developer be willing to pay $4 million? Only when demand for housing is perfectly elastic, so the demand curve is horizontal (see Fig 2a below). In this case, shifting the supply of housing doesn’t reduce the price, because supply always intersects demand at the same price. So Condon is simply assuming that upzoning does not reduce housing prices.

If we relax this assumption, then the demand curve slopes downward (as usual), and increasing supply reduces the price (Fig 2b).5 For example, if a mass upzoning increases supply and causes housing prices to fall to $900psf, then the land price residual is now $400psf instead of $500psf. Hence, developers would only be willing to pay $400*8000 = $3.2M, instead of $4M.6 Notice what happened: land values went up and housing prices went down; but according to Condon, this should be impossible.

The effect of increased housing supply for different demand curves

Another problem is that Condon is focused on the land price of a single parcel, instead of the market-wide price of each type of land. When the parcel is upzoned, it switches from 1-FSR to 2-FSR. This makes the set of all 1-FSR parcels smaller by one, and the set of all 2-FSR parcels bigger by one. So 1-FSR land becomes more expensive and 2-FSR land becomes cheaper. But Condon completely ignores this effect, because he does not track land prices by type before and after the upzoning.

Finally, this argument doesn’t even support Condon’s thesis. His claim is that higher land values are the cause of higher housing prices, and that upzoning is the main cause of rising land values. But in this example, the goalposts have shifted: now the goal is to show that upzoning does not reduce housing prices. That’s because Condon doesn’t have a model of prices; by assuming perfectly elastic demand, he effectively takes prices as fixed, and determined outside of the model. Hence, even if we set aside the reliance on a theoretical edge case, Condon is unable to support his thesis that upzoning increases housing prices.

IV. A backwards analysis of inclusionary zoning

Condon favors inclusionary zoning (IZ) as a solution to the housing crisis, because it reduces land values. Under the ‘density bonus’ version of IZ discussed by Condon (from Cambridge, MA), developers are granted permission to build extra density in exchange for making all units in the project Affordable (that is, available at subsidized prices).

Continuing the numerical example from above, if the city imposes a housing price of $750psf on the new Affordable units, then the developer’s willingness-to-pay for land is: $750psf - $500psf = $250psf. For 8,000sf (at FSR=2), this means the developer will pay up to $2M for the parcel, compared to $4M in the market-rate upzoning scenario.

For Condon, IZ reduces housing prices because it reduces land values:

In this example, if a nonprofit housing corporation or co-op is assumed, the selling price per square foot drops from $1,000 to $750, with all of this price reduction attributable to lower land cost (p.197).

But this gets the causality backwards. The lower housing price is not ‘attributable’ to the lower land price; rather, the regulated housing price directly causes the lower land price. By artificially controlling the housing price, we reduce developers’ willingness-to-pay for land, which in turn reduces the land price. So the land price is a symptom, not a cause.7

Condon mistakenly implies that IZ works like a cost-reduction policy, by reducing the price of housing inputs (namely, land). But IZ is a non-market policy. It has no direct effect on the market forces that led to a housing price of $1,000psf.8 Using IZ to reduce land costs is like dipping a thermometer in cold water to reduce the air temperature.

This is an important point. IZ doesn’t address the question of how we make market-rate housing more affordable. Even though the vast majority of people live in market housing, Condon has no answers for them. Because he rejects supply and demand, Condon has given up on fixing the housing market, and instead wants to switch to a non-market approach.

While IZ is better than doing nothing, it is still worse than simple upzoning. With Condon’s version of IZ, the supply of market-rate housing cannot respond to increases in high-end demand, because only new subsidized units are allowed. This causes a demand cascade, where unabsorbed demand at the top of the market cascades down and increases competition for low-end homes, which in turn forces poorer residents to look for non-market housing. Hence, while IZ does increase the supply of subsidized homes, it also increases the need for subsidized housing. IZ is self-defeating.9

Original meme for context

V. A failure of value capture

And we can do land value capture in a market system. Using my upzoning example above, when we relax the assumption of perfectly elastic demand, housing prices fall from $1000psf to $900psf and the land value increases from $2M to $3.2M. Since the minimum amount the landowner is willing to accept (their reservation price) is $2M, the city could impose a windfall tax of $1.2M on the landowner. In this case, the surplus created by upzoning is split between lower market-rate housing prices and higher land values, which are taxed for public benefit. Again, higher land values are consistent with lower housing prices.10

Because Condon insists that only upzoning can cause higher land values, his approach to value capture is woefully incomplete. As we’ve seen, land values in First Shaughnessy have multiplied, despite zero upzonings. This land lift is driven by increased demand, and cannot be captured by regulations that apply only conditional on redevelopment. Condon’s fixation on upzoning has blinded him to billions of dollars in land lift that should have been captured for public benefit. The solution is to tax land values unconditionally, as with a land value tax.11

Moreover, the same focus on upzoning means that Condon prescribes taxing only new housing, while leaving old housing untouched; this makes him a favorite among NIMBY activists. In fact, we can see in the graph above that the 2019 property tax coincided with a decrease in land values; but Condon appears uninterested in raising taxes that affect all homeowners.

VI. A partial grasp of development taxes

Condon does not have a model of the land market as a whole. Instead he analyzes a single parcel, which leads to a misleading analysis of development charges:

Development taxes […] will affect the residual value of each buildable square foot of a subject parcel. When development taxes are added, the land-price residual goes down, and when development costs are removed, the land-price residual goes up. In this way, development taxes do not add to housing prices for new owners but reduce the residual value per buildable square foot of the development site, thus lowering the money that a developer is able to offer the landowner (p.197-198).12

This is true for a limited context. Specifically, when the tax amount is low enough that the project remains feasible, then the developer does pay less for land. Housing prices are set by the total quantity of homes supplied and demanded; since these haven’t changed, housing prices are unaffected by the tax. So development taxes are another way to implement land value capture.

But more generally, excessive taxes can kill projects, which reduces supply and raises prices. Obviously a tax of $100M would not merely reduce the land price residual. Instead, it would lower the developer’s willingness-to-pay below the landowner’s reservation price, so the transaction would fail and the project would be cancelled. Since the tax reduces the quantity of homes supplied, it raises housing prices. In general, to evaluate the effect of a tax on housing prices, we need to look at the change in the total number of homes.13

Condon doesn’t seem to recognize the risk of killing projects:

However, a development tax imposed late in the approval process may make the business case for a project unworkable. At that point, either the project will be cancelled or the price for the land will be renegotiated down. Municipalities are therefore in a position to moderate or eliminate land-price inflation by signalling their intention, years ahead of time, to impose a development tax. One might call this process disciplining the land market. (p.163)

Again, if projects are cancelled because a tax, then supply is reduced and housing prices are higher compared to the no-tax scenario. The same is true if the tax prevents projects from being started in the first place. Condon does not show an awareness of how taxes affect housing prices, since he is narrowly focused on minimizing land values.

It’s worth spelling out what Condon’s strategy of “disciplining the land market” means. For Condon, housing does not follow supply and demand, so reducing market-rate housing prices is off the table. Instead, our only hope is to make land cheap enough for non-market projects to pencil. This requires imposing enough costs on developers that they stop building new projects, and hence stop offering to buy land at high prices. In turn, landowners would eventually realize that their land won’t sell for a gain, and that they might as well sell it to a non-market developer with a lowball offer. Hence, we get cheap land for non-market housing.14

But as I noted above, this strategy would be a disaster. It does nothing to absorb demand increases, so demand cascades would occur continuously, raising prices and pushing more and more people into needing subsidized housing. Fortunately, Condon’s views are incorrect, and we can improve affordability by upzoning to increase housing supply.

library(VancouvR)
library(tidyverse)

# time series plot of average land and building values in First Shaughnessy

# remove upzoned parcels and character infill
  # if zoning changes, then no longer FSD
  # hence, require that zoning is FSD in all years
# use assessment date: July 1 of previous year

fsd_data <- search_cov_datasets("tax")$dataset_id |>
  map_df(\(ds)get_cov_data(ds, where = "zoning_district= 'FSD'",
         select = "pid,land_coordinate,zoning_district,neighbourhood_code,report_year,current_land_value,current_improvement_value")) |>
  mutate(n=n(),.by=land_coordinate) # count number of times a parcel appears

est_data <- tibble(Date=as.Date(c("2018-02-01")), label=c("Announced"))

png(file=paste0("shaughnessy.png"), width=750, height=400, res=90)
fsd_data |>
  # keep parcels with 18 observations as FSD
  # if >18, then multiple conversion dwelling: multiple dwellings per parcel
  filter(n==length(unique(fsd_data$report_year))) |>
  summarize(`Land value`=mean(current_land_value),
            `Building value`= mean(current_improvement_value),
            .by=report_year) |>
  mutate(Date=as.Date(paste0(as.integer(report_year)-1,"-07-01"))) |>
  arrange(Date) |>
  pivot_longer(cols=c(`Land value`, `Building value`), names_to="type", values_to="value") |>
  ggplot(aes(x=Date, y=value, colour=type)) +
  geom_line() +
  geom_point(shape=21) +
  geom_vline(data=est_data, aes(xintercept=Date), colour="black", linetype="dashed") +
  scale_y_continuous(labels=\(x) scales::dollar(x, scale=10^-6, suffix="M")) +
  scale_color_manual(values=c("Land value"="#1F77B4", "Building value"="#FF7F0E")) +
  labs(
    title="First Shaughnessy land and building values",
    x="Assessment year",
    y="Average value",
    colour=NULL,
    caption="Note: extra school tax announced in 2018, implemented in 2019."
  ) +
  guides(linetype="none") +   # removes legend entry for vline
  theme(plot.caption = element_text(hjust = 0) # left-align caption
        , legend.position = "right",
        legend.justification = "center"
        ) 
dev.off()

# same graph for rest of Vancouver
nonfsd_data <- search_cov_datasets("tax")$dataset_id |>
  map_df(\(ds)get_cov_data(ds, where = "zoning_district != 'FSD'",
                           select = "pid,land_coordinate,zoning_district,neighbourhood_code,report_year,current_land_value,current_improvement_value")) |>
  mutate(n=n(),.by=land_coordinate) # count number of times a parcel appears

png(file=paste0("non_shaughnessy.png"), width=750, height=400, res=90)
nonfsd_data |>
  filter(is.na(current_land_value)==0 & is.na(current_improvement_value)==0) |>
  summarize(`Land value`=mean(current_land_value),
            `Building value`= mean(current_improvement_value),
            .by=report_year) |>
  mutate(Date=as.Date(paste0(as.integer(report_year)-1,"-07-01"))) |>
  arrange(Date) |>
  pivot_longer(cols=c(`Land value`, `Building value`), names_to="type", values_to="value") |>
  ggplot(aes(x=Date, y=value, colour=type)) +
  geom_line() +
  geom_point(shape=21) +
  geom_vline(data=est_data, aes(xintercept=Date), colour="black", linetype="dashed") +
  scale_y_continuous(labels=\(x) scales::dollar(x, scale=10^-6, suffix="M")) +
  scale_color_manual(values=c("Land value"="#1F77B4", "Building value"="#FF7F0E")) +
  labs(
    title="Vancouver land and building values (excluding First Shaughnessy)",
    x="Assessment year",
    y="Average value",
    colour=NULL,
    caption="Note: additional school tax announced in 2018, implemented in 2019."
  ) +
  guides(linetype="none") +   # removes legend entry for vline
  theme(plot.caption = element_text(hjust = 0) # left-align caption
        , legend.position = "right",
        legend.justification = "center"
  ) 
dev.off()

2

We see a similar increase in land values in the rest of Vancouver, where upzoning happens much more frequently compared to First Shaughnessy. So upzoning cannot be main explanatory factor.

4

Condon has a meme version of this example, where the key point is that housing prices are held fixed by assumption.

By the same logic, upzoning agricultural land to single-family residential doesn’t reduce housing prices, because it just increases land values.

12

Condon also has a meme version of this argument.

The key assumption is that fees are less than the land price residual; if fees are larger, then the residual is negative and the project fails.

When we upzone land from single-family zoning to apartment zoning, we change the allocation of a city’s fixed stock of land. Upzoning makes apartment-zoned land more abundant and hence cheaper, while single-family-zoned land becomes scarcer and more expensive. Because land is an input into the production of housing, upzoning reduces the cost of building apartments, and hence is a key policy option for reducing housing costs.

In this post I’ll set up a two-sector model of land and show the effects of upzoning. (If you get stuck, try reading this post first.) House developers can use either single-family-zoned land or apartment-zoned land to produce houses, while apartment developers can use only apartment-zoned land. Since it’s more lucrative to build an apartment building than a house on a given parcel, apartment developers are willing to pay more for land; as a result, the price of apartment-zoned land is higher than the price of single-family-zoned land.

Upzoning moves a parcel from the single-family- to the apartment-zoned market. This makes apartment-zoned land more abundant, reducing its price; because land is an input into the production of apartments, upzoning reduces the price of apartments. In the extreme case, if we upzone all land, then house developers are indifferent between the two types of land, and we get an equilibrium where land prices are equal across submarkets.

A two-sector model of land

Consider a representative house developer with perfect substitutes preferences over house-zoned and apartment-zoned land. For the house developer, a parcel is a parcel, since they can build a house on it regardless of the zoning. (Parcels of different zoning types are identical in size.) Hence, they treat both types of land as interchangeable, and will choose the lower-price option. To simplify, I assume they have a fixed budget to spend; in a complete model, developer demand for the input (land) is derived from the price of the output (housing). Also note that we are considering vacant parcels with no teardown cost.

Click the footnote to see the underlying demand functions.1

The graph below shows the demand curve for house-zoned land, with the price of of house-zoned land (P_H) on the y-axis. Recall that this is an inverse demand function, so we read it as: for a given price, what quantity of land is demanded? For example, when the price is 2, quantity demanded is 12.

Here, I’m showing an equilibrium where the price of the substitute good (apartment-zoned land) is P_A=12, so when P_H<12, the house developer buys only house-zoned land and spend their entire budget on it (bottom curved segment). The horizontal segment is where P_H=P_A, so the house developer is indifferent; any quantity along this segment is equally good (and the remaining budget is spent on apartment-zoned land). When P_H>12, apartment-zoned land is cheaper, so demand for house-zoned land is 0 and the demand curve overlaps the y-axis.

For the apartment-zoned land market, we have to aggregate the demand of both house and apartment developers (see graph below). Apartment developers spend their whole budget on apartment-zoned land, regardless of how cheap house-zoned land gets. Because they have higher willingness to pay, their quantity demanded is larger at higher prices. (Note that I’m using continuous quantities, which allows buying an infinitesimally-small quantity of land; in practice, the demand curve would intersect the y-axis at some price.)

As before, house developers will spend everything on apartment-zoned land if it is cheap enough (bottom curved segment in blue). If apartment-zoned land is more expensive than house-zoned land, the demand curve is 0 and overlaps the y-axis (and they go all-in on house-zoned land). Here, the equilibrium has the substitute price at P_H=2, so when P_A=P_H=2, they are indifferent between the two, corresponding to the horizontal segment.

Adding up the sector demand curves horizontally, we get aggregate demand for apartment-zoned land (see below). In the upper-left curved segment we will get a corner solution where apartment developers buy only apartment-zoned land, and house developers buy only house-zoned land. (When P_A>2, the aggregate demand curve is equal to the apartment developers’ demand curve.) The horizontal segment is the interior solution where house developers buy some apartment-zoned land and prices are equal. (The bottom-right curved segment is the other corner solution where both developers buy only apartment zoned land and P_A<P_H; I’ll discuss this case later.)

Here’s an equilibrium for the corner solution. To maintain incentive compatibility, we require that P_A>P_H, so that the house developer chooses the cheaper option (house-zoned land). This holds when the (fixed) supply of house-zoned land is large and the supply of apartment-zoned land is small.

At these prices, all supply is demanded, so this is an equilibrium. Note that we also have a market clearing condition for supply across sectors: S_A+S_H=S=13, where S is the total stock of land. (I use S=13 to make the math work.)

Small upzoning

Let’s upzone a small amount by moving one parcel from the house market to the apartment market (so S_A increases from 1 to 2 and S_H falls from 13 to 12). This shifts S_A to the right, which reduces P_A from 12 to 6. At the same time, S_H shifts left, increasing P_H from 2 to 2.2.

Because the house developer treats the two land types as substitutes, these price changes also ‘shift’ the demand curves. (Recall from last time that this is actually a movement along the 3D demand surface.) House-zoned land becomes more expensive, so demand for its substitute (apartment-zoned land) increases; this is the shift up/right in demand for apartment-zoned land (from D1 (blue) to D2 (red)).

Similarly, apartment-zoned land becomes cheaper, so demand for its substitute (house-zoned land) falls, shown by the shift down/left in demand for house-zoned land. These changes in the demand curves occur away from the intersections with the supply curves, so the equilibria are unaffected.

So upzoning makes apartment-zoned land more abundant, and the price has to fall to clear the market. This reduces the price of land, making apartments cheaper to produce. Hence, upzoning reduces the price of apartments. (The model here shows only the input market (for land), but intuitively, decreased input prices translate to decreased output prices.)

Conversely, houses become a bit more expensive, since house developers have to compete over a more limited stock of land. This is progressive, because houses are inherently more of a luxury good than apartments (ask yourself: which has more square footage, a private yard, private parking?). (Note that upzoning does not literally remove land from house developers; instead, upzoning moves land to a submarket with higher prices, where house developers are unwilling to pay the market price.)

With this model, we can see where the “but upzoning raises land costs” argument goes wrong. By changing the allocation of land across zoning types, upzoning makes apartment-zoned land cheaper and house-zoned land more expensive. While the upzoned parcel itself increases in price (from 2 to 6), that parcel was previously unavailable to apartment developers at any price. Adding it to the apartment-zoned market makes apartment developers better off, because P_A falls from 12 to 6. The fallacy in “but upzoning raises land costs” lies in looking at a single parcel instead of the market-wide price.

Large upzoning

Next let’s explore the interior solution where P_A=P_H. Here we shift supply enough that we reach the horizontal segment of the perfect substitutes demand function. In this case, P_A=P_H=P≈2.8; this is the maximum possible reduction in the price of apartment-zoned land. We can choose S_A≥4.33 and S_H≤8.66, subject to S_A+S_H=13; here I’ve used S_A=S_H=6.5.

Since house developers are indifferent between the types of land, we can be anywhere on the horizontal segment as long as market clearing holds. Because the demand curve is horizontal, these movements don’t affect prices, and any allocation of supply is an equilibrium.

Maximum upzoning

Finally, let’s show the case where the entire stock of land is upzoned for apartments: S_H=0 and S_A=S=13. This is really the same case as the large upzoning above, because we again have P_A=P_H=P≈2.8, and the only difference is that we move all of the way to the edge of the interior solution.

Note that this is the minimum P_A achievable, since all supply is allocated to apartment-zoned land. So when S is fixed, we can’t drive P_A down to the initial P_H=2, because we have to clear the market with demand from both types of developer, whereas P_H=2 comes from a market with only house developers (and hence less total demand).

When S_H=0 we can also have equilibria with P_H>P_A, but these have the same equilibrium price P_A and quantity S_A. (With larger values of P_H, the demand curve for apartment-zoned land ‘shifts’ up, but the intersection of supply and demand is unchanged.)

Discussion

A full general equilibrium model would include the output markets for houses and apartments, and show explicitly how output prices for housing are linked to input prices for land. One interesting case to look at would be where mass upzoning reduces PH and hence further reduces house developers’ demand for house-zoned land. For example, a huge increase in the supply of apartments downtown induces people to move out the exurbs.

Another extension is the N-sector model with multiple types of apartments. For example, the N=3 case could have houses, 3-storey apartments, and 6-storey apartments. This would also shed light on how development feasibility changes over time. For example, as demand for houses increases, so does the price; this raises house developers’ willingness-to-pay for land. If demand from house developers is sufficiently higher than demand from 3-storey apartment developers, then no 3-storey apartments will be built.

Notice that in the small upzoning, the upzoned parcel increases in price from 2 to 6. This ‘land lift’ is the motivation for value capture policies that tax windfall gains from upzoning. But in the large upzoning, the land lift per parcel is much smaller: from 2 to 2.8. This shows the inherent tension in trying to both (a) improve housing affordability, and (b) collect taxes from new developments. Local governments are incentivized to do small spot-upzonings to maximize land lift2, but because this limits the stock of apartment-zoned land, it comes at the expense of making apartments more affordable. Instead of taxing land only when it is developed, a better approach is to tax land unconditionally, like with a land value tax.

Another extension is to model anticipation effects, where landowners expect upzoning to occur in the future, which raises the price of house-zoned land today.

Code for graphs here.

A supply and demand equilibrium

Consumers start with a willingness to pay for Old and New apartments. They buy the housing type with the largest difference between valuation and price: v-p. To satisfy the unit demand constraint, their demand only counts in the market of the type they choose. So one rich person could buy up all of the homes, but the unit demand constraint prevents this, by requiring them to buy their highest-valued type, and removing their demand for the other types. Here, there are five consumers, and everyone value New apartments more than Old.

I’ll show an initial equilibrium where Person 1 buys a New apartment, and Persons 2 and 3 buy Old. Because Person 1 buys New, their demand is removed from the Old market. Hence, the aggregate demand curve for Old apartments looks like this:

Since each person demands one unit, aggregate demand tops out at four. Next, here’s the supply and demand equilibrium for Old apartments. The supply curve is vertical, representing a fixed stock of Old apartments. Since the supply curve overlaps the demand curve for some segment, technically the equilibrium is not well-defined. However, this is not a big deal, because we can pretend the supply curve is slightly sloping upward, so the intersection is unique (though this brings in continuous quantities). Alternatively, if we had a large number of consumers, the demand curve would be a straight line that intersects supply uniquely; working with discrete quantities is just a pain.

So Persons 2 and 3 each get an Old apartment, and the price is 2. Note that Persons 4 and 5 are initially priced-out, and don’t get a home. We can interpret this as having to move away, or living in their car (or being homeless).

Next we have the aggregate demand curve for New apartments. Here we show Persons 1, 4, and 5, because Persons 2 and 3 buy Old apartments, so their demand for New is removed (by the unit demand constraint).

In the initial equilibrium, only Person 1 gets a new apartment, and the price is 4. The supply curve is vertical, representing zoning constraints: developers cannot build any more, even if prices went up. Note that Persons 4 and 5 are priced out here as well.

Vacancy chains

To show a vacancy chain, we increase the supply of New apartments. This will allow Person 2 to upgrade, vacating their Old apartment, which in turn allows Person 4 to get a home (instead of being homeless).

The higher supply of New apartments reduces the price, as we move down the demand curve. This lower price induces Person 2 to re-evaluate their housing choice. Their willingness to pay is 3.7 for New apartments, and 2.2 for Old. The price for New apartments was 4, so they chose Old. But now the price is 2, so their payoff for a New apartment is 3.7 - 2 = 1.7, and the payoff for an Old apartment is 2.2 - 2 = 0.2 Hence, the New apartment is a better deal, so they switch markets from Old to New.

Person 2 upgrading means that we also change which market their demand contributes to. So we subtract their demand from the Old demand curve, and add it to the New demand curve, shifting it to the right. For New apartments, this pushes up the price. Note that we can end up out of equilibrium if this increase in price is not consistent with Person 2’s willingness to pay. Here, I’ll use the modelling flexibility we have from the supply and demand curve overlapping to choose a new equilibrium price of 3.

Since Person 2 upgrades, we remove their demand for Old apartments, shifting it left. Now Persons 3 and 4 are able to get an Old apartment, and the price falls to 1.6. (That is, supply intersects demand on the section of the demand curve where Persons 3 and 4 buy.) This is the vacancy chain in action: Person 4 was initially priced out and didn’t have a home, but by adding a New apartment, we have freed up an Old apartment for Person 4 to live in!

To make sure this is an equilibrium, let’s check the payoffs. Person 1 is happy to buy New, because they get 4.7 - 3 = 1.7 from New and 2.6 = 1.6 = 1 from Old. Person 2’s payoffs are 3.7 - 3 = 0.7 (New) and 2.2 - 1.6 = 0.6, so they buy New. Person 3 gets 2.2 - 3 < 0 for New and 2 - 1.6 = 0.4 from Old, so they buy Old. So each consumer’s choice is consistent, hence, these prices define an equilibrium.

Note that with unit demand, we are able to have consumers being priced out (like Person 4 initially). With continuous quantities, this was not possible, since they could always buy some infinitesimal quantity, and so can’t be priced out completely. But with discrete units, Person 4 initially doesn’t get anything, then thanks to the vacancy chain, they get an Old apartment.

Demand cascade and yuppie fishtank

Now let’s see how demand cascades and yuppie fishtanks work in the unit demand case. Since people can be priced out entirely, demand cascades will push people out of old apartments, and yuppie fishtanks will allow them to avoid being pushed out.

We’ll use the same willingness to pay as before, but this time we’ll start with Person 1 living outside of the city. Person 2 starts in a New apartment, so they don’t contribute to demand for Old apartments. That leaves Persons 3, 4, and 5 making up the aggregate demand curve for Old apartments.

There are two Old units, so the equilibrium has Person 3 and 4 each getting an Old unit at a price of 1.6.

For New apartments, Persons 2 and 5 contribute to the demand curve.

But there’s only one New unit supplied, so only Person 2 gets a New unit in the equilibrium. The price is 3, which ensures that everyone is choosing their best option. Person 5 is priced out of both housing types, and has to live in their car.

The demand cascade starts with Person 1 moving in and bidding for a New apartment. This shifts the demand curve to the right (from D1 to D2a), increasing the price from 3 to 4.

Since Person 1 is richer, they outbid Person 2 for the New apartment, so now Person 2 downgrades to an Old apartment. Hence, we remove their demand from the New demand curve; this does not affect the new equilibrium. Now only Persons 1 and 5 contribute to the aggregate demand curve.

Turning to Old market, we add in Person 2’s demand, shifting the demand curve to the right. This pushes up the price, from 1.6 to 2. Now Persons 2 and 3 get an Old apartment, and Person 4 is priced out and doesn’t have a home. This is the demand cascade: when high-income Person 1 moves in, they bid up the price of new apartments, leading Person 2 to downgrade to an old apartment, which leaves Person 4 being priced out. In short, when supply is restricted, rich people moving in forces poor people to move out.

But what if supply is not restricted, and grows to match the increase in demand? This is the yuppie fishtank: by building a new glass tower to contain Person 1, we absorb their demand and protect the old apartments from competition. So we shift supply from 1 to 2, reducing the price.

This lower price induces Person 2 to upgrade back to a New apartment. So the demand curve shifts right, and the price is 3. This is the same as the initial equilibrium: the yuppie fishtank has absorbed Person 1’s demand, keeping prices flat.

And since Person 2 is not competing for an Old apartment, the demand curve shifts left. Thanks to the yuppie fishtank, Person 4 gets to stay in their Old unit, and isn’t priced out.

Notice that a yuppie fishtank can cancel out a demand cascade caused by a rich newcomer, while a vacancy chain triggered by a rich local upgrading is an inverse demand cascade.

See here and here for code to produce the graphs.

When we build new apartments, the people moving in vacate their old apartments; this reduces competition for old housing, making it more affordable. This is the vacancy chain effect: building expensive new housing helps improve affordability of cheaper old housing.

When rich people move into a city, they outcompete locals for new homes; those locals in turn compete for the stock of old homes, raising prices for poor people. This demand cascade is the mechanism driving the housing crisis. We can reverse demand cascades by building yuppie fishtanks to absorb the demand of rich newcomers.

In this post I’ll show how to use supply and demand to model vacancy chains, demand cascades, and yuppie fishtanks. I explain how to formalize the key mechanism in these scenarios, where changes to one market have effects on another market. This post treats quantity as continuous (so people can demand 1.26 homes, say) to show how standard microeconomics applies to housing; a future post will consider discrete quantity and unit demand.

A supply and demand equilibrium

Let’s consider an example of n=3 consumers with perfect substitutes preferences. People choose between Old and New apartments, and have a housing budget to spend. They differ in how much they prefer New vs Old apartments, and in the size of their housing budget. With perfect substitutes, you choose one good or the another, and spend your entire budget on the chosen good. For example, Person 1 will buy Old if price(Old) is less than half of price(New). Conversely, they buy New if price(New) is less than double price(Old). (We use two housing types as the simplest model that demonstrates the key mechanism; in reality, there are dozens of quality gradations.)

Here are the individual demand curves for Old apartments. Preference for Old (vs New) is decreasing from Person 1 to Person 3 (since Person 1 chooses an Old apartment at the highest price) while budget size is increasing (at price=2, Person 3 demands the largest quantity). Since New apartments are better, everyone gets more utility from a New apartment compared to an Old one, but they differ in how much more.

People choose between Old and New apartments based on a threshold determined by their preferences and the ratio of prices of Old and New apartments. Demand for Old apartments is equal to 0 above the threshold (here the demand curve overlaps the y-axis), is indeterminate at the threshold (the horizontal segment), and is equal to budget divided by price below the threshold (recall the shape of a 1/x function). Technically, the demand function has a discontinuity at the threshold, so you can imagine the horizontal part as being a dotted line. Also note that I’m referring to quantity demanded as a function of price, even though price is on the y-axis. Here I’m following the convention to plot price against quantity.

For example, Person 1 (blue) buys an Old apartment when the price is less than 9. Their quantity demanded is equal to their income (10) divided by the price; at price=2, this is 10/2=5. When the price is above 9, they demand 0 Old apartments and instead choose a New apartment; we’ll see that when we plot the demand curves for New apartments. When the price is equal to 9, their demand curve has a discontinuity and jumps across the horizontal segment.

Note that demand for Old apartments is a function of the prices of both Old and New apartments, but I’ve plotted it using only price(Old). What’s going on? The short answer is that I’m showing you one slice of the entire demand function, evaluated at a particular value of price(New). I’ll return to this later on.

Click the footnote to see the underlying demand functions.1

What does it mean to demand multiple apartments? A more complicated model would have unit-demand (ie. people demand either one Old or one New apartment), but it’s simpler to use continuous quantity to start. For now, you can interpret it as the square footage of the apartment, which is a continuous variable.

We get aggregate demand by summing up the individual demand curves: for each price, count the quantity demanded across consumers. This produces downward-sloping segments where we can identify which consumers contribute to aggregate demand. (Recall that the horizontal segments are discontinuities, so I have the arrows pointing at the sloped portions of the curve.) Since Person 1 has the strongest preference for Old apartments, they show up at the top of the demand curve, while Person 3 (with the weakest preference) shows up only at the bottom.

Next, let’s see the individual demand curves for New apartments. These are the reverse of demand for Old: because Person 1 had the strongest preference for Old (vs New), they have the weakest preference for New (vs Old). Again, budget size is increasing from Person 1 to 3. And keep in mind that these demand curves are evaluated at a specific value of price(Old).

Here’s aggregate demand for New apartments. Now Person 3 has the strongest preference, so they contribute to aggregate demand in all segments, while Person 1 contributes only in the bottom segment.

An equilibrium is prices (P_Old,P_New) that set quantity supplied equal to aggregate quantity demanded in both markets. This is where the demand curve being evaluated at the price of the other good is key: the equilibrium price in one market has to be consistent with the equilibrium price in the other. I’ll show an equilibrium where Person 2 initially buys an Old apartment. To capture the vacancy chain effect, we’ll increase supply of New apartments, lowering the price, which induces Person 2 to upgrade from Old to New and reduces demand for Old apartments, thereby reducing pressure in that market.

Here’s an equilibrium for Old apartments. To capture the stock of Old apartments being fixed, I use a vertical supply curve; no matter what the price is, developers can’t add any more old apartments (at least in the short run). I’ve chosen the supply curve so that Person 1 & 2 buy Old, while Person 3 buys New (ie. we’re on the second segment of the demand curve for Old apartments). The price is set by the intersection of the supply and demand curves; here, P_Old=2.3.

Here’s the corresponding equilibrium for New apartments. In this case, the supply curve is upward sloping, reflecting that developers are willing to build more units at higher prices (and that building them is permitted by zoning, which may be an unrealistic assumption). Supply intersects demand on the segment of the demand curve where only Person 3 buys a New apartment. Here the price is PNew=10.

You can confirm that these prices are the ones I used when plotting the demand curves for the other good. For example, the first graph shows the individual demand curves for Old apartments when price(New) = 10.

When a consumer buys one good, what happens to their demand for the other one? With perfect substitutes preferences, the demand is still there, but it can never be realized, because perfect substitutes involves a knife-edge threshold. For Person 2, if price(Old) < price(New)/4, they buy Old; otherwise they buy New. Here, that is 2.3 < 10/4 = 2.5, hence they buy Old. If the prices were different in a way that flipped the inequality, they would buy New. (But to be an equilibrium, the supply curve would have to match the quantity demanded.) In the next post on unit demand, I’ll show how this logic changes.

Stepping back, note that these demand functions are actually 2D slices of a 3D demand surface. Demand for Old apartments depends on the prices of both Old and New apartments, so it makes sense that the demand curve will be plotted as a function of two variables (the two prices). Here’s a plot of an individual demand surface (for Old apartments). The demand curves plotted above are slices of this surface, taken at specific values of price(New). If you rotate your head to the left, you can see it: when price(Old) is high, demand is 0 (overlapping the y-axis); the original horizontal discontinuity becomes vertical here in 3D; and as price(Old) goes to 0, quantity becomes large, following the 1/x shape.

Changing the price of the other good moves you along the surface, but in 2D this will look like shifting the demand curve. This is important, because changing the cross-price does not shift demand in the sense of changing preferences. Instead, we’re evaluating the same demand function at a different price. For example, moving from price(New)=8 to price(New)=4 does not represent a change in preferences, but the demand curves (evaluated as slices at those prices) will look different, reflecting different tradeoffs (i.e., as New apartments get cheaper, my demand for Old apartments falls).

Vacancy chains

Let’s see how a vacancy chain works. The idea is that building new apartments will induce people to upgrade from Old to New, thereby reducing demand for Old apartments and making them more affordable.

We start the chain by increasing the supply of new apartments. For example, upzoning reduces land costs, which makes developers willing to produce more at every price. Shifting from S1 to S2, we move down the demand curve, with a new price P_New=3.

Note that now supply intersects demand in the segment where both Person 3 and 2 buy New. Induced by the lower price of New apartments (P_New falls from 10 to 3), Person 2 has upgraded from Old to New! So we should see a corresponding drop in demand for Old apartments.

Here’s the effect on Old apartments. The demand curve ‘shifts’ down from D1 to D2, and now supply intersects demand on the segment where only Person 1 buys Old. The price falls from P_Old=2.3 to P_Old=0.8. Following the intuition for substitute goods, when price(New) falls, Old apartments become relatively less attractive, so demand for Old apartments decreases.

As mentioned above, this is not really a shift in demand, but a movement along the 3D demand surface caused by changing price(New). Person 2’s preferences haven’t changed, they just make a different choice at a different price.

So we’ve shown the vacancy chain effect: increasing the supply of new apartments reduces their price (from 10 to 3), which induces people to upgrade from old to new apartments, thereby reducing demand for and the price of old apartments (from 2.3 to 0.8). Thanks to the new apartments being built, Person 1 now gets all 13 of the Old units to themself, and at a lower price.

But if price(Old) fell, shouldn’t that reduce demand for New apartments too? Yes. These are substitute goods, so a reduction in the other price makes a good less appealing. In this case, demand for New apartments ‘shifts’ down, but not enough to change the equilibrium. Hence, there are no more changes, and we’ve converged on a new equilibrium, with (P_Old,P_New) = (0.8, 3).

If we added more consumers, we could show how the vacancy chain allows a poorer person to afford an Old apartment at the new lower price, when they were originally priced out. And with more housing sub-markets (with multiple degrees of quality, instead of only Old vs New), we could trace out the vacancy chain itself, with someone upgrading in each submarket and reducing demand for their original housing type, thereby enabling someone in the submarket below to upgrade.

Demand cascades and yuppie fishtanks

Now let’s look at demand cascades and yuppie fishtanks. When rich people move into the city, they increase demand for expensive new apartments. This pushes locals to downgrade, increasing demand for old apartments. The end result is higher prices for both old and new housing; sound familiar?

But when we increase the supply of new apartments to match the rise in demand, we can prevent the demand cascade and stop the price of old apartments from going up. This is a yuppie fishtank, where we build shiny glass towers to contain the rich yuppies moving in and absorb their demand for housing.

To illustrate, I’ll show an example with two people forming an initial equilibrium, which is disrupted by a rich person moving in.

Click the footnote to see the underlying demand functions.2

Here are Person 1 and Person 2’s demand curves for Old apartments. As before, Person 1 has a stronger preference for Old apartments (relative to Person 2), and Person 2 has a larger budget.

We sum their demand curves to get aggregate demand.

Now let’s see the demand curves for New apartments. In this case, Person 3 is much richer and has a very strong preference for New apartments.

But they will move in later, so the aggregate demand curve does not include them (yet).

Here’s the initial equilibrium for Old apartments. Supply intersects demand on the segment where only Person 1 buys. The price is P_Old=0.8. As before, the supply curve is vertical, representing a fixed stock of Old apartments.

And here’s the initial equilibrium for New apartments. Supply intersects demand on the segment where only Person 2 buys, and the price is P_New=3. So we have Person 1 buying Old and Person 2 buying New.

Next, the rich Person 3 moves in. This shifts the demand curve for new apartments up (from D1 to D2a), resulting in a new equilibrium at P_New=10. But now supply intersects demand on the segment where only Person 3 buys, so Person 2 has been priced out and will downgrade to an Old apartment.

The increased price of New apartments makes Old apartments more attractive, so demand for Old apartments shifts up. Now supply intersects demand on the segment where both Person 1 and Person 2 buy, and the price rises from 0.8 to 2.3. The stock of 13 Old units is now shared by both Person 1 and 2. This is the demand cascade: rich people moving in and increasing competition in the market for new apartments results in higher prices of old apartments, making housing less affordable for the poor. (This is also known as up-filtering, where for a given home, poorer residents are replaced by richer residents.) Here I’ve shown how demand cascades over two steps; in reality, demand would flow down dozens of steps, representing different levels of housing quality.

To finish up the new equilibrium, we account for the effect of the higher price(Old) on the New market. Higher prices for Old apartments make New apartments more attractive, so demand shifts up (from D2a to D2b), but this doesn’t change the equilibrium.

Now let’s use a yuppie fishtank to reverse the effects of the demand cascade. We’ll increase the supply of new apartments enough to offset the increased demand from Person 3. Supply shifts from S1 to S2, and the price falls from 10 back to 3. Now supply intersects demand on the segment where both Person 2 and 3 buy, so Person 2 has upgraded from Old to New (just as in the vacancy chain).

Since Person 2 upgrades, demand for Old apartments goes back to the original level, and the price falls to 0.8. This is the yuppie fishtank: by building new apartments, we absorb the demand of rich newcomers and prevent prices of old housing from rising.

Tidying up, the fall in price(Old) shifts demand for New apartments down, but the equilibrium is unchanged.

To conclude, note that vacancy chains and yuppie fishtanks are closely related. A vacancy chain allows a local resident to upgrade, reducing demand for old housing. A yuppie fishtank absorbs demand from a rich newcomer, preventing a demand increase for old housing. And the reversal tests work as expected. Reducing the supply of new apartments (say, a new building burns down) causes a demand cascade, while reducing demand for new apartments (say, deporting yuppies) creates a vacancy chain.

See here and here for code to produce the graphs.

Atwood (2022) studies the long-term effects of the measles vaccine on employment and income. Barteska et al. (2023) uses a similar approach for the effects on education. Both papers find positive effects, which makes a consistent story about the mechanism: contracting measles causes children to miss school, which reduces human capital, which reduces adult income. So the vaccine increases education, which increases adult income and employment.

One difference between these papers is that Atwood uses 1964 as the treatment year (the vaccine was introduced in 1963, so this should be 1963), while Barteska et al. emphasize that vaccine takeup was low until the 1967-68 immunization campaign. So in a sense, these papers are contradictory: if Atwood finds a treatment effect in 1964, then there are pre-trends in Barteska et al.’s specification. Atwood calculates measles incidence over 1952-63, while Barteska et al. use 1963-66, so the treatment groups could be different.

Both Atwood and Barteska et al. have a graph showing the decline in reported measles incidence after the vaccine.

This is taken as evidence that measles incidence declined more in the high-measles states. But this is also consistent with actual incidence being the same across states, and reporting capacity being different. Since measles was eradicated to 0, each state’s reported incidence will be reduced by ~100%. So this graph is also consistent with the story that there is no geographic variation in actual measles incidence.

In my comment on Atwood (2022), I ran an event study and found post-vaccine trends: the treatment effect is increasing for cohorts born after the vaccine, which doesn’t make sense, since there’s no variation in the treatment dose (vaccine access). Barteska et al. (2023) does run an event study, and doesn’t find the same post-vaccine trends. My event study, following Atwood, used a 16-year treatment window over 1949-1964, with comparison windows 1932-1948 and 1965-1980. Barteska et al. use an 8-year window over 1959-1966, with comparison windows 1950-1958 and 1967-1975. Note that in the Barteska et al. event study, the coefficients start increasing 8 years before the immunization campaign. In my event study for Atwood, the coefficients start increasing 16 years before the vaccine. If Barteska et al. are right that the vaccine was targeted towards younger children, then the treatment effect for ages 9-16 (at vaccine introduction) is not consistent with a vaccine effect.

What treatment variation is Barteska et al. using, if there’s no long-run geographic variation in actual incidence? Since they use a shorter window for calculating measles incidence (1963-66), it’s possible they are measuring differences in short-run incidence, based on differences in epidemic cycle timing. In contrast, Atwood calculates average measles incidence over 12 years, which would average out differences in cycle timing. So Barteska et al. could be capturing variation in the susceptible population; i.e., children who avoid measles and the negative effects of disease, and get vaccine-induced immunity instead of virus-induced immunity.

Chuard et al. (2022) extends this reasoning to explicitly capture variation in the susceptible population. They use an epidemiological SIR model to directly measure geographic variation in actual cases before the vaccine. This method should work for evaluating the long-term effects of the measles vaccine, since cohorts with low incidence get vaccinated and avoid disease, increasing their education and adult income. But they find no effects, while a model using pre-vaccine measles mortality finds effects similar to Atwood. They note that Atwood is using a different source of variation compared to the epidemiological model, but it’s still not clear exactly what that is: what is the difference between High- and Low-measles states?

Given the similarity between their results and Atwood’s, they conclude that Atwood’s variation (reported incidence) is a proxy for disease severity. But this treatment variation is ambiguous, because disease severity can be affected by worse viral outbreaks (e.g., getting a worse case of measles from a higher viral dose), initial health levels (the same viral dose causes worse disease in sick people), or health infrastructure (the same viral dose causes worse disease when health care is poor). The concept of ‘disease burden’ doesn’t seem to apply for a universal disease like measles. They also don’t discuss differences in health infrastructure leading to different reporting rates.

Moreover, if the results using mortality and incidence are similar, then presumably the pre-vaccine state averages should be correlated. They do not report this correlation. But Figure 4 shows share ever-infected and mortality for a sample of five states, and the correlation is negative:

WA has the highest share ever-infected and the lowest mortality, while AR has the lowest share ever-infected and the highest mortality. Also, if incidence is a proxy for disease severity, then it should be correlated with worse health outcomes.

Chuard et al. also make a few odd choices in research design. Like Atwood, they don’t run an event study to test for pre- or post-trends. They say (footnote 5) that they calculate birthyear using age and survey year, even though IPUMS has a birthyear variable. They also aggregate the data to the state-of-birth X year-of-birth X age level, instead of using the micro data and clustering standard errors.

There are several other papers on measles vaccines. Atwood and Pearlman (2023) does the same exercise as Atwood (2022), but for the 1973 vaccine in Mexico. This also faces the problem of long-run average incidence being ambiguous treatment variation. Noghanibehambari (2023) copies Atwood (2022), with birth outcomes as the dependent variable. Berg et al. (2023) studies the UK measles vaccine using the same identification strategy as Atwood, and finds no average effect on height or education.

Time series regressions

Cook has two measures of patents per year: (1) using the year the patent was applied for, and (2) using the year the patent was granted. In the paper, Figure 1 reports Black (and white) patents per million using grant year, while Figure 2 shows Black patents per million using application year. Comparing the two graphs, we immediately see that the scale differs by a factor of about 10. Here I merge the two datasets and plot the application-year and grant-year variables on the same graph.

There is a huge discrepancy between the two patent variables. Cook collected data on 726 patents over 1870-1940, but the average by grant-year is 0.16, while the average by application-year is 1.22.^{1 2}

Cook’s replication data does not include the raw patent or population variables, so we can’t say for sure what’s going on here. But the average Black population (see Table 1) was roughly 10 million, and 0.16 grant-year patents/M * 10M * 71 years = 114, far fewer than the 726 patents recorded. In contrast, 1.22 application-year patents/M * 10M * 71 years = 866, which is in the ballpark of 726. Speculating, one possible explanation is that Cook calculated grant-year patents using the white population (average 75 million) in the denominator, giving 0.16 * 75 * 71 = 852 patents.³ Hopefully Cook will publish the raw data and we can resolve this.

In any case, the grant-year patent variable seems clearly flawed, while the application-year variable looks correct. Since the Table 6 results use the grant-year patent variable, we should run a robustness check using the application-year variable.

Table 6 uses time series data to estimate the effect of lynchings, riots, and segregations laws on patents. Column 1 uses race-year panel data, where the lynching and patent variables vary by race (but the riot and segregation law variables vary only by time). Columns 2 and 3 run time series regressions separately by race, allowing us to estimate differential effects of racial violence on patenting.

I am able to reproduce Table 6⁴, using grant-year patents:

As noted in the paper, lynchings and riots have negative effects on Black patenting, and the 1921 dummy has a large negative effect, corresponding to the Tulsa Race Riot.

For the robustness check, I redo Table 6 using application-year patents instead of grant-year patents. This specification actually seems more appropriate, since Cook’s mechanism is that racial violence deters innovation by Black inventors; so racial violence would first impact patent applications, and with a lag impact granted patents. So the effects should be stronger using the application-year variable.

The application-year variable is missing in 1940, which reduces the sample size for the robustness check by 1. To make a pure comparison, I re-run the grant-year regressions dropping 1940, and get similar results (see footnote⁵). Next, I run Table 6 using application-year patents:

The results are dramatically different: the negative effect of lynchings and riots disappears, as does the negative effect in 1921. If the grant-year patent variable is incorrect and the application-year variable is correct, then the paper’s main result is wrong.

Panel data regressions

In Tables 7 and 8, Cook uses state-level panel data over 1870-1940 to run regressions of patents on lynching rates, riots, and segregation laws. However, we can immediately see a problem: there are 49 states and 71 years in the data, but only N=430 observations. A complete, balanced panel would have 3210 observations, as the number of states grows from 38 in 1870 to 49 in 1940 (including DC; see code for details). So Cook is using 430/3210 = 13% of the full sample.

And the pattern of missing data is not random. Below I plot the number of observations by state and year. First, we see that the majority of states have fewer than 10 observations over 71 years.

Next, the sample size is increasing up to 1900 before dropping off and rising again starting in 1920.

Decomposing by region, we see that the Midwest and Mid-Atlantic regions are relatively overrepresented, while the South and West are relatively underrepresented.⁶

Moreover, consider how this imbalanced panel compares to the full time series. There are 726 patents in the time series, and 702 in the panel data (for 97% coverage). But the violence variables are drastically under-reported: there are 35 riots in the time series data, but only 5 in the panel data (14%). Similarly, there are 290 new segregation laws in the time series data, but only 19 in the panel data (7%).⁷ (The same problem applies with lynchings, but the replication files don’t have count data, so we can’t quantify it.)

What explains the missing data? It appears that Cook dropped any state-year observation that had a variable with a missing value. The resulting dataset has no variables with missing values, but a lot of missing state-year observations, and hence a severely imbalanced panel.

With this low level of data coverage, I’m skeptical of the panel data results in Tables 7 and 8. It’s possible that these results are unbiased, and would remain stable as the missing data was filled in (through a law of large numbers argument). Especially considering the prior plausibility that racial violence and patents are negatively correlated, we should place some weight on this.

But it’s also possible that they’re false positives. And statistically significant results are easy to get when you’re working with small effects and noisy data. For example, let’s check for heterogeneous effects by region; a robust result should be stable across different cuts of the data. From Table 7, I run the Column 1 regression separately for each region:

The lynchings estimate for the South (-0.075) is similar to the average effect from the full sample (-0.058). But there’s no estimate at all for the Midwest and Northeast, since there were zero lynchings in those regions. The estimate for the Mid-Atlantic is huge with two stars, 200x bigger than the South estimate. But this is almost certainly a Type M error (an overestimate of the true effect), as the lynching rate for the Mid-Atlantic is 3% of the average.

With only 5 riots in the dataset, it’s no surprise that there’s no estimate for the Midwest, Northeast, or West regions (which had zero riots in this data). The effect size is somewhat similar for the South and Mid-Atlantic, perhaps indicating a more homogeneous effect of riots on patenting.

For segregation laws, the Table 7, Column 1 estimate is -0.1. The effects for the South and West are in the ballpark, at -0.19 and -0.16. But the effects for the Midwest and Mid-Atlantic are positive, massive, and have three stars! But statistical significance doesn’t mean anything here, because the data is noisy. There are 19.33 new segregation laws in the data, with 17 in the South, 1 in the Midwest, 1 in the West, and 0.33 in the Mid-Atlantic (presumably a data error).

Another way to assess noisy data is to decompose the patenting variable by economic category. In fact, Cook does this in Table 8, running separate regressions for assigned patents (e.g., the patentee sells their patent to a firm), mechanical patents, and electrical patents (note that mechanical and electrical patents can be assigned or not).⁸

(For comparison, the Table 7, column 1 estimates are: lynchings -0.058***, riots -0.429***, segregation laws -0.1.) The lynching estimates are much smaller than in Table 7, and none are statistically significant.⁹ The riot estimates have the same sign and similar magnitude only for assigned patents. For segregation laws, the coefficient has the opposite sign for assigned, double the magnitude for mechanical, and half the magnitude for electrical patents. Overall, there is strong heterogeneity in the effects of racial violence, and Cook does not provide a theory to predict the pattern of varying estimates. This heterogeneity is more consistent with noise than a clear causal effect.

My takeaway from these subsample results is that the missing data is causing low statistical power, and we’re seeing Type S and Type M errors. Hence, we shouldn’t place much weight on the correlations in Tables 7 and 8, since they would probably change considerably if we had a complete and balanced panel.

Conclusion

To summarize, the main time series result in Cook (2014) is not robust to using an alternative patent variable, and the panel data results are questionable because of missing data. Nonetheless, the conclusions remain plausible, because they have a high prior probability. Lynchings, race riots, and segregation laws were a severe problem, and it would be astonishing if they didn’t have pervasive effects on the lives of Black people.

But with the data available, it’s unrealistic to think we can statistically detect causal effects. Credible causal inference would require more complete data as well as an identification strategy more convincing than a panel regression (not to mention modelling temporal and spatial spillovers). Descriptive analysis is the most that this dataset can support, and is a valuable contribution in itself, along with the rich qualitative evidence in the paper.

Cook deserves credit for pursuing this important research question and putting in years of effort to collect the patent data. And in fact, recent research, no doubt inspired by Cook, does find that segregation (of the federal government by Woodrow Wilson) and riots (specifically, the Tulsa Race Massacre) had substantial negative effects on Black Americans. I hope that more researchers continue in Cook’s footsteps and bring attention to the consequences of America’s racist history.

In terms of computational reproducibility, Cook’s code has several problems:

• The code for Figures 1, 2, and 3 is in Stata graph editor format, which cannot be run from a do-file.

• Figure 1 uses the variable patgrntpc, patents by grant-year per capita, but the graph refers to patents per million. Similarly, Table 5 reports ‘Patents, per million’, but the code uses patgrntpc. The variable should be named ‘patents by grant-year per million’.

• There’s no code for Table 4.

• Equation 1 and Table 6 refer to patents per capita, but the variable in the code, patgrntpc, has mean values of 0.16 for Blacks and 425 for whites; this is patents per million, not per capita.

• The code for Table 6 refers to a variable LMRindex, but the dataset contains DLMRindex.

• Section 3.2 mentions that the state-level regressions use data over 1882-1940, but the code uses data over 1870-1940.

• The code for Table 7 includes a command to collapse the data down to the state-year level, but the data is already in a state-year panel.

• The code for Table 7 includes a variable, estbnumpc, for the number of firms per capita, but it is not included in the dataset.

• The code for Column 1 in Table 7 includes the ‘number of firms’ variable, but the paper only includes it in columns 3-6.

• In the notes to Tables 7 and 8, Cook writes that “Standard errors robust to clustering on state and year are in parentheses.” However, the code only clusters by state, using vce(cl stateno).

• The code for Table 8 has an error in its clustering command, using the incorrect syntax vce(stateno) instead of the correct vce(cl stateno).

• The code for Table 8 does not exactly reproduce the results in the paper. When I run the code, I get N=429, while Cook’s regressions have N=428. It’s possible that Cook is controlling for firms per capita, as in Table 7, but this variable is not included in the code, and is not mentioned in the table.

• The code for Table 9 does not reproduce the results in the paper.

There are also a few data errors:

• State 9 has the South dummy equal to 1 for all years, but also has the Mid-Atlantic dummy equal to 0.33 in 1888.

• State 14 has the Midwest dummy equal to 1 in all years except 1886, when both it and the South dummy are 0.5.

• State 31 in 1909 has a value of 0.333333 for ‘number of new segregation laws’, which should be integer-valued.

Footnotes

See here for code.

1. Cook notes that “a comparison of a sample of similar patents obtained by white and African American inventors shows that the time between patent application and grant for the two groups was not significantly different, 1.4 years in each case.” (p.226, fn. 15) Also, there is no application-year patent data for 1870-72. ↩

2. This discrepancy becomes even more puzzling when we compare the paper and the code:

   • Figure 1 reports patents per million by grant year, but uses a variable named patgrntpc with the label ‘Patents by grant year’. The ‘pc’ would seem to indicate patents per capita.

   • Figure 2 reports patents per million by application year, using a variable pat_appyear_pm, with ‘pm’ corresponding to ‘per million’.

   • Table 5 presents descriptive statistics, with a ‘Patents, per million’ variable with a mean of 0.16, but the code uses patgrntpc.

   • Equation 1 and Table 6 both refer to patents per capita. The code for Table 6 uses the logarithm of patgrntpc.

   Although the variable patgrntpc would seem to be ‘Patents by grant year, per capita’, this can’t be true: the average value is 0.16 for Blacks, and 425 for whites. These values are clearly measured per million. So the variable must be misnamed, and actually represents patents per million, as described in Figure 1 and Table 5. This means that Equation 1 and Table 6 are mistaken: the dependent variable is log patents per million, and not log patents per capita. ↩

3. Another explanation is that the application-year variable counts all patents that were applied for, including patents that were denied. This is not consistent with the text, where Cook only mentions 726 granted patents. In footnote 15, Cook writes that analyzing “[a]pplication rejection rates […] is beyond the scope of the current paper.” Moreover, even if true, this explanation doesn’t account for why the grant-year variable does not add up to 726. ↩

4. Cook’s Table 6 incorrectly shows the lynching estimates in Columns 2 and 3 as having p-values less than 0.05. ↩

5. Note that N = 110 and 55 instead of 112 and 56.

   

    ↩

6. Number of states by region: South 15, Midwest 12, Northeast 6, West 12, Mid-Atlantic 7. Eleven states enter after 1870, and hence have fewer than 71 years in the complete panel. See code for details. ↩

7. The actual number is 19.33. Somehow, one state-year observation has a value of 0.33 for the number of new segregation laws. ↩

8. In Column 4, Cook runs a regression using Southern patents as the dependent variable. That is, while still using the full panel, the patent variable is set to 0 for non-Southern states. This is an incorrect approach for estimating heterogeneous effects. A correct approach would restrict the sample to Southern states, as I did above, or use the full sample and interact the violence variables with a South dummy. ↩

9. Cook mentions in footnote 49 that lynchings have a negative effect on ‘miscellaneous patents’, but this is not reported in the table, and the variable is not included in the dataset. ↩

In this post I replicate the paper “Is Legal Pot Crippling Mexican Drug Trafficking Organisations? The Effect of Medical Marijuana Laws on US Crime” by Gavrilova, Kamada, and Zoutman (Economic Journal, 2019; replication files).

I find three main problems in the paper:

• it uses weighting when its own justification doesn’t apply

• it uses a level dependent variable, and isn’t robust to log-level or Poisson models

• it does not test for pretrends in the disaggregated crime variables, and two alternative event studies show that the results are driven by differential trends

Introduction

This paper studies the effect of medical marijuana legalization on crime in the U.S., finding that legalization decreases crime in states that border Mexico. The paper uses a triple-diff method, essentially doing a diff-in-diff for the effect of legalization on crime, then adding an interaction for being a border state.

The paper uses county level data over 1994-2012, with treatment (medical marijuana legalization, MML) occurring at the state level. The authors use “violent crimes” as the outcome variable in their main analysis, defined as the sum of homicides, robberies, and assaults, where each is measured as a rate per 100,000 population. They also perform separate analyses for each of the three crime categories.

The basic triple-diff regression is:

\[y_{cst} = \beta^{border} D_{st}B_{s} + \beta^{inland} D_{st} (1-B_{s}) + \gamma_{c} + \gamma_{t} + \varepsilon_{cst}.\]

Here \(y_{cst}\) is the outcome in county \(c\) in state \(s\) in year \(t\); \(D_{st}\) is an indicator for having enacted MML by year \(t\); \(B_{s}\) is an indicator for bordering Mexico; \(\gamma_{c}\) are county fixed effects; \(\gamma_{t}\) are year fixed effects. The full model also includes time-varying controls, border-year fixed effects, and state-specific linear time trends. The outcome is crime rates per 100,000 population, measured in levels, so the regression coefficients will not have a percentage interpretation; we’ll come back to this later.

This isn’t a standard triple-diff. In this model, \(\beta^{border}\) is capturing the absolute effect of MML in border states, and not the differential effect relative to inland states. To see this, compare to:

\[y_{cst} = \beta^{DD} D_{st} + \beta^{DDD} D_{st} \times B_{s} + \gamma_{c} + \gamma_{t} + \varepsilon_{cst}.\]

Here, \(\beta^{DD}\) represents the effect of MML in inland states, and \(\beta^{DDD}\) is the differential effect in border states (relative to the effect in inland states). That is, \(\beta^{inland} = \beta^{DD}\) and \(\beta^{border} = \beta^{DD} + \beta^{DDD}\). This is perhaps an issue of taste. What I would primarily want to know is whether MML had a larger effect in border states relative to inland states; the absolute effect in border states is secondary. Hence, I will report results from the second model (although the differences are small, because the inland effect is small: \(\beta^{inland} = \beta^{DD} \sim 0\)).

The authors find that, on average, MML reduces violent crimes by 35 crimes per 100,000 population, but the estimate is not statistically significant (the standard error is 22). Then, zooming in on the border states, they find a significant reduction of 108 crimes per 100,000 (and a nonsignificant increase of 2.8 in inland states). There are three border states that legalized medical marijuana: California, New Mexico, and Arizona. (Texas is the remaining border state.) Splitting up the effect by treated border state, we have a reduction of 34 in Arizona, 144 in California, and 58 in New Mexico.

I don’t really like this “zoom in on the significance” style of research. We can always find significance if we run enough interactions. And as we zoom in on subgroups, we lose external validity: can we make meaningful predictions for a state or country that was legalizing marijuana and didn’t border on Mexico? Moreover, the identifying assumptions become harder to believe. When n=3, it’s more plausible that differential shocks are driving the result (compared to n=20, say). That is, it could be that crime was already decreasing in the three border states when they passed MML, and the negative correlation between MML and crime is coincidental.

Ok, let’s get into the issues.

Weighting

The authors use weighted least squares (weighting by population) for their main results. They justify weighting by performing a Breusch-Pagan test, and finding a positive correlation between the squared residuals and inverse population. This implies larger residuals in smaller counties. In other words, there is heteroskedasticity, and weighting will decrease the size of the standard errors, i.e., increase precision. However, in Appendix Table D7, you’ll note that while they get a positive correlation when using homicides and assaults as the dependent variable, this coefficient is negative and nonsignificant for robberies. So by the Breusch-Pagan test, the robbery results actually should not be weighted. And in Table D9, the unweighted robbery estimate has smaller standard errors than the weighted one: weighting is reducing precision. And yet, the paper still uses weighting when estimating the effect of MML on robberies (in Table 4). We’ll see below that this makes a big difference for the effect size.

Modelling the dependent variable

The authors estimate the effect of MML on crime using a level dependent variable instead of taking the logarithm, which I had thought was standard. In particular, their main results use the aggregate crime rate, which leads to a “level” interpretation: MML reduces the crime rate by \(\hat{\beta}=\) 108 crimes per 100,000 population.

I would have used a log-level regression, taking \(log(y+1)\) for the dependent variable (adding 1 if there are zeroes in the data), which gives a percentage (or semi-elasticity) interpretation: MML reduces crime by \(100 \times (exp(\hat{\beta})-1) \%\). The paper doesn’t justify why they don’t use a log-level model. This is even more surprising when you see that they manually calculate the semi-elasticity (p.19), again without mentioning the log-level approach.

After spending some time looking into this question of logging the dependent variable for skewed, nonnegative data, I’m still pretty confused. It seems the options are: (1) level-level regression, as used in this paper; (2) log-level regression; (3) transforming \(y\) with the inverse hyperbolic sine; and (4) Poisson regression (with robust standard errors, you don’t need to assume mean=variance). But it’s not clear what the “correct” approach is. I’d expect a true result to be robust across multiple approaches, so let’s try that here.

I estimate the triple-diff model using a level-level regression (to directly replicate the paper), a log-level regression, and a Poisson regression. (The inverse hyperbolic sine approach is almost identical to log-level, so I skip it here.) To see how the specification matters, I conduct a specification curve analysis using R’s specr package. Specifically, I run all possible combinations of model elements, either including or excluding covariates, population weights, state-specific linear time trends, and border-year fixed effects. This will allow us to see whether possibly debatable modelling choices, such as state-specific linear trends, are driving the results.

Here are the homicide results, first in the level-level model (as in the paper). Panel A plots the coefficient estimates in increasing order, while panel B shows the corresponding specification. Each specification has two markers in panel B, one in the upper part indicating the model, and one in the lower part indicating whether all or no covariates are included in the model.¹ For example, the specification with the most negative estimate is ‘trends + weights, no covariates’. In both panels, the x-axis is just counting the number of specifications, and the color scheme is: (red, negative and significant), (grey, insignificant), (blue, positive and significant). The ‘baseline’ specification omits the state-specific trends, border-year fixed effects, and doesn’t weight by population. I’ll be focusing on the full specification, ‘trends + border + weights, all covariates’, which includes state-specific linear trends, border-year fixed effects, and weights by population.

Level-level model: homicides

We can see that the estimate is negative and statistically significant in the full specification, with and without covariates. Most estimates are nonsignificant; these are generally the unweighted models, indicating the importance of population weighting for these results.

Log-level model: homicides

Next, in the log-level model, most estimates are insignificant, including the full specification. Two models even have positive and significant results (in blue). Let’s see the Poisson model:

Poisson model: homicides

Here I use the homicide count (instead of the rate per 100,000 population), though note that the controls include log population and I’m weighting by population. In this case, the estimates are almost exactly zero and nonsignificant in the full specification. So, the homicide results only go through using the level-level regression, and not in the log-level or Poisson models.

For the other dependent variables, I’ll show the graphs in the footnotes. The results for robberies are more robust. The full specification is negative and significant across all three models.² However, the assault results are not robust, with the full specification nonsignificant for both log-level and Poisson regressions.³

This doesn’t look great for the paper. I’d expect real effects to be robust across the three models. I conclude that at best, the paper provides evidence for an effect of MML on robberies in border states, but not on homicides or assaults. And this is assuming the event study graph looks good for pretrends, which I’ll discuss next.

Event study

There are big trends in crime over this period. Crime fell a lot during the 90s, and again after 2007. To show that their results aren’t driven by these trends, the authors present an event study graph in Figure 6, estimating a triple-diff coefficient in each year. Basically, this is estimating the triple diff for each year relative to an omitted period.

The authors estimate their main results using the 1994-2012 sample. For the event study, they also use an extended sample from 1990-2012. The extended sample has issues, because it uses flawed imputed data over 1990-1992, and the year 1993 is missing entirely. Here I will show results from the main sample, 1994-2012.

For their main event study, the authors only include dummies for relative years -2 to 4, and bin all years 5+ in one dummy. This is because California is treated in 1996 and only has two years of pretreatment data, and wouldn’t contribute to any dummies before -2.⁴ But this is a bit of an arbitrary choice. Similarly, Arizona is treated in 2010 and only has two years of post-treatment data, and hence doesn’t contribute to any dummies after +2. So should we include dummies only for [-2,2]?

I think it’s fine to include dummies for [-5,5], with the understanding that some states do not contribute to some estimates. (Specifically, California doesn’t have dummies for -5 to -3, and Arizona doesn’t have dummies for 3 to 5+.) In this setup, the omitted years are <-5, in contrast to the standard approach of omitting relative year -1. (As noted in the last footnote, California has no omitted years, so the software should drop one year.)

Next I plot my version of their event study graph, using a level dependent variable. Since this is a triple-diff, I include relative year dummies for the treated states, as well as separate relative year dummies for the treated border states. I plot the coefficients on the border-state relative year dummies. While the paper only includes dummies for [-2,5+], I estimate coefficients for [-5,5+].⁵

Event study: violent crimes (binning 5+)

Compared to the event study in the paper, here the coefficients are all negative. For the pretreatment estimates (treatment occurs in period 0), this means level differences between the treated border and inland states. There is also a slight downward trend before the treatment, hinting at differential trends.

In any case, note that this graph is for the aggregated violent crime variable. Where are the event studies for the individual dependent variables? The authors do not show them! This is a major flaw, and I can’t believe that the referees missed it. Even if we found no pretrends in the aggregate variable, there could still be pretrends in the component variables. Let’s take a look ourselves.

Event study: homicides (binning 5+)

First up, using the homicide rate as the dependent variable, we get a big mess. There are big movements in years -3 and -2: relative to the treatment year, homicides were higher three years prior, and lower two years prior. So at least for homicides, it looks like the negative triple-diff estimate could just be picking up noise. Now we know why the authors didn’t include separate event study graphs by dependent variable.

Event study: robberies, unweighted (binning 5+)

For robberies, recall that the Breusch-Pagan test failed to justify weighting, so I do not use weights. Here, it also looks like a negative trend is driving the result: robberies were smoothly decreasing in treated border states before MML was implemented. (See the unweighted graph in the footnote.⁶) The common trends assumption for the triple-diff appears to be violated.

Event study: assaults (binning 5+)

Finally, for assaults, the event study actually doesn’t look bad, although the standard errors are large. This is a bit surprising, given that the assault results were not robust across log-level and Poisson models.

Overall, this doesn’t look good for the paper. I think this is an equally defensible event study method, but it nukes their homicide and robbery results.⁷

I’m not a fan of binning in event studies. In Andrew Baker’s simulations, binning periods 5- and 5+ performs badly. In contrast, a fully-saturated model including all relative year dummies (except for relative year -1, which is the omitted year) performs perfectly. So let’s try that here.

By omitting year -1, we’re basically normalizing the above event study graphs around the -1 estimate (but also changing the estimates, since we’re including all other relative year dummies). Hence, the homicide graph has the same patterns, but shifted up. We again find a clear trend in the robbery graph. But now the assault graph also looks to be driven by trends.

Event study: homicides

Event study: robberies

Event study: assaults

Takeaway: now I really doubt that MML had a causal effect on crime.

Synthetic control

To further dig into these trends, I aggregated the data from county- to state-level and performed a synthetic control analysis for each of the three treated border states: California, Arizona, and New Mexico. This aggregation is probably imperfect, and it would be better to start with state-level data, but let’s see what happens. (Running level-level regressions, I still get negative results, with effect sizes similar to the county-level data. See the specification curves in the footnote. ⁸)

The idea of synthetic control is to construct an artificial control group for our treated state, so we can evaluate the treatment effect simply by comparing the outcome variable in the treatment and synthetic control states. The synthetic control group is a weighted average of control states, and these weights are chosen to match the treated state on preperiod trends. I use the nevertreated states as the donor pool; I’ll report the weights below.

Here I’ll show the robbery results for the three states (using the level dependent variable), to see what’s happening with that smooth trend. Note that these graphs are plotting the raw outcome variable, so we’re seeing the actual trends in the data.

California’s synthetic control is 68% New York and 28% Minnesota. California’s MML occurs in the middle of the 1990s crime decrease, and it doesn’t look like there’s much of an effect in 1996.

Arizona’s synthetic control is 61% Texas, 24% Florida, and 15% Wyoming. Again, there doesn’t seem to be a treatment effect.

New Mexico’s synthetic control is 51% Mississippi, 21% Louisiana, 18% Texas, and 7% Wyoming. Its MML occurs before a drop in robberies that is partly matched by the synthetic control group.

You can look at the other synthetic control graphs in this footnote.⁹

Overall, I worry that these three states coincidentally legalized medical marijuana when crime was high and falling, and that the triple-diff estimates are just picking up these trends. Based on my analysis here, I don’t believe that medical marijuana legalization reduced crime in the US.

Randomization inference

One final note: the paper calculates a (one-sided) randomization inference p-value of 0.03, and claims that this is evidence for their result being real. However, as I discuss in this post, this claim is false. With large sample sizes, there’s no reason to expect RI and standard p-values to differ, so a significant RI p-value provides no additional evidence.

Conclusion

I think it’s plausible that moving marijuana production from the black market to the legal market would reduce crime (at least in the long run). But the effect of medical marijuana legalization on crime is too small to detect in the data.

Footnotes

See here for R code, and here for the original replication files. (For some reason, the replication files aren’t online anymore.)

PS: Table 5 does heterogeneity by type of homicide; I’d be curious to see the event study for each of these outcomes.

1. The full covariate list is: an indicator for decriminalization, log median income, log population, poverty rate, unemployment rate, and the fraction of males, African Americans, Hispanics, ages 10-19, and ages 20-24. In general, I find that adding controls barely changes the \(R^{2}\), so these variables aren’t adding much beyond the county and year fixed effects. ↩

2. Robbery results:

   Level-level model: robberies

   

   In the level-level model, we see a big difference between the weighted and unweighted results. Clearly, there are heterogeneous treatment effects, with larger effects in the higher-weight states (California, probably). As I noted above, the robbery estimates should not be weighted.

   Log-level model: robberies

   

   Poisson model: robberies

   

    ↩

3. Assault results:

   Level-level model: assaults

   

   Log-level model: assaults

   

   Poisson model: assaults

   

    ↩

4. One problem with this specification is that California has no omitted years. Every year from 1994-2012 has a dummy variable, which seems like a dummy variable trap (i.e., multicollinearity). Specifically: 1994-2000 are covered by dummies for -2 to 4, and 2001-2012 are covered by the 5+ binned dummy. ↩

5. Moreover, as noted above, I am estimating the differential effect of MML in border states relative to inland states, while GKZ are estimating the absolute effect. I also drop counties that have the black share of population greater than 100%. It seems the authors were doing some extrapolation that got out of control. ↩

6. We shouldn’t care about this graph, because weighting is unwarranted.

   Event study: robberies, weighted (binning 5+)

   

    ↩

7. It’s depressing that event studies can differ so much based on slight model changes. I have a feeling that a lot of diff-in-diffs from the past twenty years are not going to survive replication. ↩

8. Specification curve for state-level results:

   Level-level model: homicides

   

   Level-level model: robberies

   

   Level-level model: assaults

   

    ↩

9. Synthetic control results for homicides and assaults.

   
   
   
   
   
   

    ↩

In this post, I’ll give a quick demo of regression weights, looking at Cook (2014) (sci-hub) (replication files) on the effect of racial violence on African American patents over 1870-1940. This paper starts with striking time series data on patents by African American inventors. In Figure 1, we see a big drop in black patents around 1900. What is driving this pattern?

Cook argues that race riots and lynchings cause reduced patenting, directly by intimidating inventors, and indirectly by undermining trust in intellectual property laws (if the government won’t punish race rioters, why should you believe it’ll enforce your patents?).

Table 7 contains the main state-level regressions of patents on lynching rates and riots. Using a random-effects model, Cook finds negative effects for both lynchings and riots. I find similar results with a fixed effects model.

Let’s do regression weights, first for the lynching result. I regress lynchings on the other variables, grab the residuals, square them, then normalize by the sum of squared residuals.

* Stata code:

use pats_state_regs_AAonly, clear

reghdfe lynchrevpc riot seglaw illit blksh regs regmw regne regw , ab(stateno year1910 year1913 year1928) vce(cl stateno) res(resid)

gen res1 = resid^2
egen resid_tot = total(res1)
gen regweight = res1/resid_tot

Next, let’s see how these weights vary by region.

table region, c(sum regweight count patent)

This is a bit surprising. The South has 81% of the weight, with the remainder coming from the West. The other three regions have basically zero contribution to the lynchings coefficient.

So let’s see what’s happening in the data.

table region, c(mean lynchrevpc)

It turns out that basically all lynchings occurred in the South and West, with zero in the Midwest and Northeast (and roughly zero in Mid-Atlantic). Given this, the regression weights make sense. When there’s no variation in a variable, it should contribute nothing to the regression. But because the Midwest and Northeast have data on the other covariates, they still add some information, which is why the weights aren’t exactly zero.

Next, let’s see the results for the effect of riots on patenting. First, the regression weights, regressing riots on the other controls:

reghdfe riot lynchrevpc seglaw illit blksh regs regmw regne regw , ab(stateno year1910 year1913 year1928) vce(cl stateno) res(resid2)

gen res2 = resid2^2
egen resid_tot2 = total(res2)
gen regweight2 = res2/resid_tot2

table region, c(sum regweight2 count patent)

Again, the regional patterns are surprising. This time, the South has 27% of the weight, and the Mid-Atlantic has 73%, with the other regions contributing nothing. What’s going on?

gen region = .
replace region = 1 if (regs)
replace region = 2 if (regmw)
replace region = 3 if (regne)
replace region = 4 if (regw)
replace region = 5 if (regmatl)

label define reg_label 1 "South" 2 "Midwest" 3 "Northeast" 4 "West" 5 "Mid-Atlantic"
label values region reg_label

table region, c(sum riot)

It turns out there are only 5 riots in the state-level data. Let’s dig deeper.

table stateno, c(sum regweight2 count patent)
table year, c(sum regweight2 count patent)

* output omitted

The regression weight is concentrated on three states: state 39 has 58%, state 44 has 27%, and state 33 has 15% (the names are not in the data). It’s also concentrated on four years: 56% on 1917, 12% on 1918, 15% on 1900, 12% on 1906. This is because there are five riots occurring in four years, with two in 1917 in state 39, two in state 44 in different years, and one in state 33. So the riot effect is driven almost entirely by the four state-year observations that had riots.

But wait. If you look, you’ll see that there are 35 riots in the time-series data.

use pats_time_series, clear
collapse (sum) riot if race==0
su riot

Where did the other riots go? It looks like the state data just has a lot of missing observations, which would explain the missing riots. That is, the issue isn’t variables with missing values, but that most state-year observations do not even have a row in the data. (I emailed Cook to ask about this, but didn’t get a response.) As you can see, the sample size fluctuates over time; this is far from a balanced panel.

Note that 1917 and 1918 have the majority of the weight, but there are only two observations in each of those years.

The paper is not very clear about this. Table 5 reports the descriptive stats, but only has the riots variable for the time-series data, and not the state-level data. And Cook does not plot any of the raw state-level data, but instead jumps right into the regressions.

This seems like a serious problem for the riot results. The paper isn’t estimating the effect of riots on patenting; instead, it’s doing the effect of five specific riots. If we could collect data on the remaining 30 riots, I’d expect the estimate to change. In other words, why should we expect this result to be externally valid for other historical riots?

To sum up, regression weights are an easy way to dig into a paper and see exactly what’s driving their results.

Happy replicating!