UP504 • Survey Research last updated: Saturday, April 12, 2008 link to survey research page

ANSWER SHEET DRAFT

Assignment Four due Monday, March 17

Please write concise answers. Good survey research requires clarity, precision and consistency. See the survey research page (including the detailed class notes) for more information, including the course readings on survey research. As in previous assignments, work in teams of two students.

1. Measures: In urban research we often use concepts that are not easily measurable, and yet measures are necessary for quantitative urban research. Define simple measures of the following two concepts. For each, you may either locate and use standard measures (such as government definitions or accepted social science practices -- be sure to cite the source) or else develop your own. IMPORTANT: In either case, be specific and operational. Don't just describe the measure, but concisely explain how it is actually measured/calculated. (In other words, the reader should know what data is needed, what calculations would be performed on the data, what the scale is, and how to interpret the results -- e.g., does a higher number represent more or less segregation?)

Also: if you are measuring a complex, multi-faceted concept, you may need to choose between two approaches: (a) developing a complex, multi-faceted measure (e.g., an aggregation of multiple simple measures, where you will also need to decide how to aggregate and weight each component), or (b) simply present a series of component measures and leave it up to the user/reader to decide how to aggregate.

a. racial segregation

b. suburban sprawl

2. Sampling Scenario (answer limit: 1 page)
You are a graduate planning student writing a thesis on New Urbanism (a contemporary planning/design movement to build higher density communities that encourage greater walking and mass transit, promote more neighborhood identity, reduce the environmental impacts of urban development, etc.). You want to do a survey of home builders in the U.S. to examine their attitudes about New Urbanism. You are particularly interested in whether potential support for New Urbanist developments among the construction industry varies regionally, by size of builder, and by the housing type they construct. You contact the National Association of Home Builders (NAHB), which has over 65,000 members nationwide. They have a membership list for sale, which includes names, addresses, phone numbers, size of business, predominant type of housing constructed (e.g., single family, townhouses, etc.) and average sales price of housing units constructed. However, they want to charge you $20,000 for the list. After much negotiation, they agree to give you the names of 200 members for free. (This makes you happy, but you have concerns about the sampling error when making inferences from small subpopulations of your sample.) You can specify what criteria the data base manager at NAHB will use to select the 200 cases from the mailing list data base of over 65,000 names.

Your task: Develop a sampling strategy. How would you select your sample, and why?

Your approach will depend on whether you make the assumption that you can know the characteristics of the list as a whole.

If you DO assume that you know the characteristics of the list: then stratified sampling is the way to go. Knowing the breakdown of the 65,000 members by characteristics would help you identify what subgroups are relatively small (and thus would benefit from "over sampling" using stratified sampling to ensure a critical mass of respondents in each subgroup, such as builders of high-density, "New Urbanist" communities). The trick is how many dimensions to use to select strata: e.g., geography, size, type and cost of housing. It may not be easy or necessary to stratify along ALL these four dimensions -- especially with only 200 cases to use. Select what you think are the one or two key characteristics, such as geography (e.g., region) and housing type. It is easier to select strata if one has a good sense of what strata lead to the greatest variation of the dependent variable in question.

(If you don't make this assumption about knowing the characteristics of the list as a whole, then stratified sampling may be less effective because you don't know the relative size of each population strata within the population (65,000) as a whole.)

Remember: stratified sampling can have at least two benefits: (1) ensuring that relevant subgroups in the sample have enough respondents to allow for statistically significant conclusions about that subgroup; (2) reducing sampling error (especially when subgroups are relatively more homogenous than the population as a whole: that is, variation is greater across subgroups than within subgroups).

Note: for questions 3 and 4, consider not only the specific wording/formatting, but also the broader issues of categories, concepts vs. measures, scale, measurement units, any conflicts between the intent of the question and the way the question was asked, etc.

3. Survey Questionnaires (Mail)
For each of the following mail survey questions, briefly explain what is wrong with the question (if anything), and how it might lead to biased, inaccurate or otherwise poor results. Then suggest your own version of the question: 

a. What is your occupation now, and what was it five years ago? __________________________________
 
  Three major problems:

1. two time periods asked -- separate into two questions.

2. Use filtering questions to filter out those who were not employed now or five years ago.

3. Many will confuse occupation with industrial sector, so best to use standard Census occupational categories -- either on the questionnaire itself (as a closed-ended question) or as a separate coded list.
   

b. How do you travel to work each day?
 

	___	car
	___	bus
	___	subway
	___	other

1. use a filtering question for those who don't work -- or work at home (and good to know the difference between these two)
2. use more options: e.g., walk, carpool, etc.
3. allow for modal splits (i.e., combination of several modes)
4. many commute trips involve walking (e.g., walking to the bus stop, even walking from the central city parking garage to the office). How will you treat walking as a component of an overall journey, especially if it is a secondary aspect?
5. clarify "each day"-- what to do with people who vary their mode: e.g., ask their most common / typical mode, or "how did you get to work yesterday?" etc.

c. Would you be willing to ride mass transportation if it was available in your area? (Circle one number)

very willing						not willing at all
1	2	3	4	5	6	7

1. Is the ordered scale the best approach? Perhaps it is too abstract. Instead, the responses will likely be more useful and accurate if alternatives/trade-offs presented.
2. time and space frames are too vague
3. what kind of transit, cost, alternatives? One should offer more specifics. e.g., bus, light-rail, trolley, etc.
4. One will need to know more about the person's current transportation choices and options. (this would involve two stages: several preliminary questions about current habits and options, and then questions about anticipated behavioral changes to changes in mass transit options).
   

4. Survey Questionnaires (Telephone)
After conducting a telephone survey of 400 people, you get the following results. How could you have changed the question to get more useful results?
 
 

QUESTION: If gas prices went up $2.00 per gallon, how much less would you use your car?

___________________________
 
 

[a range of answers were given, including more than one answer; these varied responses were grouped together as the following:]
 

	about the same	42%
	a bit less	27%
	between 10 and 30 percent less	12%
	less weekend trips	7%
	take public transit more to work	17%
	carpool	6%
	more than 30 percent less	17%
	give up driving	5%
	sell car	3%
	less summer trips	6%
	I would trade in my big car for a small one	9%
	I would look for a job closer to home	4%
	I would look for housing closer to work	2%

1. The disparate responses suggest that one should use a more closed-ended format: e.g., asking about either travel mileage changes (though that might be too abstract), or which trips might be forgone, etc. There are several dimensions of answers -- pick one, or use several questions to get at each dimension.

2. Distinguish between short-term and long-term: in the short-term one might simply absorb the additional costs (and spend less elsewhere), in the medium term one might change travel behavior, and in the long term one might buy a different kind of car (or even move closer to work). The extent of behavior changes will be a function of the respondent's expectations about the future price of fuel (e.g., is the price increase simply a temporary spike, or the start of a long-term increase in prices?).

3. This question is -- for an economist -- going after price elasticity with respect to demand. It might also be useful to know and specify the current price of gas -- and therefore be more clear about a $2 increase from what current level?

In conclusion, greater precision and a more structured question will lead to more consistent and useful answers.