Assignment One: Critically Reading Economic Development/Impact Studies

modified: Thursday, September 21, 2017

Goals of the Assignment

• to gain an initial understanding of local economic development studies that attempt to measure economic impacts (including costs and benefits), evaluate the effectiveness of development programs, and/or justify public subsidies, etc.
• to get a sense of the range of methods and data sources used.
• to identify terminology used in economic development research (e.g., direct vs. indirect impacts; zero-sum game; multiplier; etc.)
• to dig deeply and critically into a selected local economic development study
• Note: many of these methods and data sources may be new to you, so do the best you can for this short assignment early in the semester.

Steps of the Assignment

1. Locate an example of a local or regional economic development study/report. (Examples: an economic impact study; a marketing study; a labor impact study; etc. Ask me if you need help finding something.)
2. Carefully read and probe the assumptions, methods, sources and conclusions of the study.
3. Write a concise 1-2 page critique of the report. (Given the short page length, focus on the core issues.) Please upload to Canvas.
4. Sept 28 (in class): Make a concise, 3-5 minute presentation that summarizes the key question and method of the study, and the strengths and weaknesses of the study. Optional: you may prepare 1-2 slides to aid your presentation. Here is the link to upload presentation slides.
   
   

Here are suggested questions or issues to consider:

1. What is the purpose (stated or implied) of the study?
2. Who is the intended audience?
3. Differentiate between studies that ask a genuinely open question ("Was this program worthwhile and should we refund it for another five years?") vs. studies that seem to be rationalizations (justifications) of decisions already decided ("What are all the benefits of this project that justify its expense?")
4. Some studies ask a specific question (e.g., will a public-subsidized stadium generate new jobs and revenues?); others ask a broader question (what is the most effective use of public funds to increase local employment and revenues?). Do these two approaches often lead to different results?
5. How do the studies address time? e.g., short-term vs. long term costs and benefits?
6. How do they employ discount rates (if at all)?
7. Does the development attract new residents (from outside the city, or perhaps outside the region) vs. attracting residents from within?
8. Where relevant, how do the authors use (and make assumptions about) such terms as multiplier, direct vs. indirect vs. induced benefits, etc.
9. What assumptions do the authors make about larger economic trends (e.g., growth in demand, interest rates, etc.)?
10. Do the authors clearly explain the sources of data used in the study? Is this data publicly available?
11. Do they assume new economic activity (i.e., a net gain) when the program/policy/event/etc. might simply shuffle existing activity around (i.e., a zero-sum game)?
12. Is there evidence of double-counting benefits and/or omitting costs (such as opportunity costs)?
13. Does the report consider equity impacts? impacts on local residents vs. new arrivals? etc.
14. Does the report consider environmental impacts?
15. Note:  examples might include studies of public projects (e.g., infrastructure spending, a new bridge, government research facilities, job training program, etc.) or of public-subsidized private developments (e.g., of a stadium, casino, research park, film industry subsidies, tax increment financing, etc.).  Such studies might look at either the fiscal impact (costs and revenues for governments) and/or economic impact (jobs, wages, R&D investment, etc.).
16. If you were to do a detailed, rigorous critique of the study (perhaps attempting to replicate the results), what skills, methods or data would you need? Does your preliminary examination of the study (for this class exercise) highlight areas of economic development methods you hope to master? [Since we are doing this brief exercise at the beginning of the semester, many of the terms and methods in the study understandably will be unfamiliar to you. Some of these we will cover throughout the semester, while others may be beyond the scope of this course.]

----

Terms, Methods, Data Sources often cited:

• opportunity costs (though often ignored)
• counterfactual (sometimes ignored; sometimes assumed to be no change in activity)
• IMPLAN (IMpact analysis for PLANning) -- a proprietary input-output model.  Combines input-output analysis with social accounting matrices (SAM) and multipliers.
• BLS -- Bureau of Labor Statistics
• ACS -- American Community Survey (from the US Census Bureau)
• leakage (money “leaking” from the local economy)
• multipliers
• FDI - foreign direct investment
• import-substitution
• export-based (linked to multipliers)
• NAHB - National Association of Home Builders
• LIHTC - Low-Income Housing Tax Credit
• direct vs. indirect (impact on suppliers) vs. induced (increase in hhd income leading to spending) benefits
• double counting (e.g., counting the same benefit twice, such as increased business sales and retail spending on the same goods or services).
• “Study area”
• Local vs. exports
• Spin-offs
• "Stimulate" (the economy) versus "drive" (the economy) versus "impact" (the economy)
• “Stimulus effect”
• Direct, indirect and induced impacts
• Input-output analysis
• Multiplier effect
• Economic impact / economic activities
• Ripple effect
• Marginal increase (net impact) versus total/gross
• Quantifying non-financial benefits
• Uncounted benefits vs. non-monetary benefits
• Public amenity vs. revenue

Overall, what question are they asking – and answering?

• Did the project/program deliver what it promised?
• How many jobs, profits, government revenues, etc. did it deliver?
• Should we spend our money on X?
• We already spent all this money on X.   How can we feel good/justified about this big expense?
• If we don’t spend our money on X, what should we spend it on?
• Who benefits, and who pays?
• We do all this great work for the community (bringing arts, or education, or sports, or events, or…), but do we also bring economic activity to the community?
• What are the hidden costs of doing X?
• What are the opportunity costs?
• Why we should continue investing in X?
• Can we find enough examples of other (important, successful) cities doing X (or spending money on X) to justify that we, too, should do X?
• How can we change/improve policy?
• Can the investment in X diminish the impact of Z?

• A macro study of many diverse cases (e.g., the economic impact of all stadiums) without disaggregating by case.
• Makes assumptions without providing rationale for them
• Over-extrapolation of benefits or costs without attempting to quantify qualitative information
• Does not cite sources (or includes source, but no details)
• Only includes gross revenue (not net revenue and associated expenses)
• Potential conflicts of interest: Who pays for the study and how are their interests linked to the study outcome?
• The type of data used (does the data answer the question or is it "forced" to answer it?) and the sample size (is the sample appropriate for the size of the study?)
• The study is mathematically complex and/or data rich but not necessarily thorough or balanced or an accurate reflection of the issues.
• Does not consider rival explanations of positive outcomes (e.g., the benefits would have happened even without the intervention)

A partial list of characteristics of a good, trustworthy, useful economic impact/evaluation study.

• Transparent (e.g., conducted by a third party evaluator, not funded by group being studied)
• Clear methodology/terms used
• Clear process of calculation
• Reliable sources (for data)
• Identifying costs (or the “bad things” about X) and attempts at rebuttal
• Includes purpose / what study will look at
• accurate and accessible data visualization.
• Considers both quantitative and qualitative impacts
• Outlines assumptions and limitations

You might find these pages useful

• Understanding Economic Analyses and Models (EPA)
• ECONOMIC IMPACT CONCEPTS (MSU)
• Economic Development Research Group: Impact Studies
• National League of Cities: Your Local Economic Strengths and Weaknesses • Economic Development
• Cities Alliance: Understanding Your Local Economy: A Resource Guide For Cities

a few background readings to provide examples (see also the syllabus for more resources):

Note: please let me know (on this page and elsewhere) if you encounter broken links

• CSL International on behalf of the Detroit Riverfront Conservancy, Economic Impact Study Detroit Riverfront 2013
• Community Land Use + Economics Group, LLC, Retail Market Analysis for Downtown Atlantic City
• Partnership for a New American Economy + Center for American Progress, The Economic Benefits of the DREAM Act
• Los Angeles County Economic Development Corporation, Los Angeles Union Station Master Plan: An Economic Impact Analysis
• Ninth Annual Economic Impact Report of Michigan's University Research Corridor
• Meter, K. and Goldenberg, M.P. (2015). Critical Analysis of Economic Impact Methodologies. In Exploring Economic and Health Impacts of Local Food Procurement. Chicago, IL: Illinois Public Health Institute. [link]
• American Booksellers Association: Local First and Economic Impact Studies
• UM's Center for Labor Market Research: Economic Impact Studies
• Stephen Sheppard, Measuring the Economic and Social Impacts of Cultural Organizations
• Economic Impact of Big Ten Football Games in Michigan

------