Before my article was published The Washington Post responded to emailed questions about its poll. I found the Post's responses disappointing in the extreme and responded in an email point by point--however, there was not enough room for my responses in The Nation article, so I am sharing it here.
From: Jacqueline Keeler
Date: May 24, 2016 at 6:46:45 PM PDT
To: "Clement, Scott" <scott.clement@washpost.com>
Cc: Dave Z.
Subject: Re: Questions
Thanks Scott, I appreciate your answers. The U.S. Census does allow self-reporting of Native American identity but its own population estimates of Native Americans are 29% lower. Should 29% be struck from the total?
Misreporting, whether purposefully or not, is very common when it comes to Native American ancestry. It is not always mean-spirited or malicious but the result of the history of this country, genocide and the romanticization of Native American people that has occurred. See how Elizabeth Warren has been lambasted for proudly believing a family story of Cherokee ancestry that it turned out had no basis in fact.
Also, you did not check to see what respondents understood “enrollment” to mean. Their understanding of the terms provided is essential to the value of their answers. They may have assumptions that are not what you expect.
Also, your number of respondents is too small to allow for oversampling or undersampling by age or region.
Fully 50% of the Native American population is under the age of 29. Native Americans have the lowest life expectancy—Native American men in particular. How can you account for, even with weighting, fully 54% of the respondents being over the age of 50?
As only 76 respondents were under the age of 29 and 274 were over 50 years old, how many more phone calls would you need to make to achieve a significant number of respondents under 29? I understand 28,000 calls were made to achieve the 76. Social scientists I spoke to felt that 1,500 respondents would provide a better picture. Since it took 28,000 calls to receive 504 self-reported Native Americans, wouldn’t that require 90,000 or more phone calls? And even then, that would only at this rate provide you with 228 respondents under the age of 29. Doesn’t it seem like there are more effective ways to reach the Native American community?
Also, 35% of the respondents were from the South--an area with few Native Americans since most of the tribes were force marched out of the region during the Trail of Tears in the 1830s. This seems like a gross oversampling. And the percentage of college graduates (25%) does not match that of the Native American population—9%. It’s 150% more.
Professor Fenelon’s study was a qualitative one and not a quantitive one, this is true. But social scientists have developed accepted mathematical tools to analyze the data. His work was presented at the American Sociological Association and qualitative surveying is regarded as acceptable in social science as is quantitative methods like phone calling. Perhaps more so in this case because with your survey you have no idea whether the respondents are truly Native or not. Also, the 12 you chose to feature in your paper, was a qualitative sampling of data obtained quantitatively. This is not acceptable social science practice without clearly stating the methodology for selecting these 12 and giving real reason why to provide more weight to their opinions.
Your response to NAJA’s statement, which clearly cites the APA resolution still ignores the harm mascotting at this level causes to Native Americans and to all Americans (as recent studies have shown-University of Buffalo 2015) and is to abrogate a trust to minimize harm to your readers in your coverage. An example of this care can be seen in the coverage of individual suicides. Studies show that such coverage causes a copycat effect and so most newspapers do not cover suicides individually to protect their readership. Likewise, the science clearly demonstrates the negative, cumulative effect of mascotting on the Native American population. A Stanford study also found that Native Americans who claimed to be okay with mascotting actually suffered measurably greater loss of self-esteem after being exposed to Native American mascots than those that said they were not okay with such mascots—the exact opposite of the inference you have drawn from your survey. How can you ignore findings that directly negate the very meaning of your findings?
-Jacqueline
On May 24, 2016, at 6:07 PM, Clement, Scott <scott.clement@washpost.com> wrote:
Dave and Jacqueline,
Below are our responses to your questions.
Best,
Scott
1) Why no verification of tribal enrollment, just taking people at their word that they are Native American?
The survey of Native Americans was conducted as part of five months of ongoing weekly surveys of U.S. adults conducted on cellular and landline phones by Social Science Research Solutions, a firm in Media, Pa. During those surveys, which interviewed more than 25,000 adults overall, respondents were asked what race they consider themselves. The 504 individuals who identified as Native American were immediately asked our poll’s questions about tribal enrollment, the Washington Redskins’ team name and Native American imagery in sports.
Self-identification is a survey technique that is accepted, common and time-tested in measuring personal attributes -- in political polls as well as official surveys covering a wide range of other issues. The Census Bureau uses self-identification to measure race, and this method is also used in surveys reporting on health and other characteristics of the Native American population.
The Post survey used self-identification as a starting point for compelling methodological reasons. This approach made it possible to compare the demographic makeup of the survey’s sample with Census Bureau statistics, allowing us to identify and correct for differences, which is a best practice in the survey industry (Described here). Using self-identification was also important to conducting a systematic national sample of the population, 95 percent of which can be reached through conventional or cellular telephones.
To analyze whether attitudes differed among relevant subgroups, the Post poll asked respondents whether they were members of a tribe and, if so, which one. It also collected information on whether respondents lived on or near reservations. As it turned out, there was little variation in the responses to questions about the Redskins’ team name given by those who said they were enrolled in a tribe and those who said they were not. Similarly, there was little variation in responses from those who lived on or near reservations and those who did not.
We see little reason to suspect respondents would intentionally misreport their racial identity or tribal status to a confidential survey. The overall poll results suggest respondents understood a distinction between racial identification and tribal membership, given that a majority of self-identified Native Americans said they were not enrolled. The substantive questions about the team’s name came at the end of polls on other subjects and after survey respondents already had self-identified, leaving no motive (or even opportunity) for individuals to self-identify as Native American when they learned that the questions would center on the team’s name.
2) Given how young the Native American community is, why speak to no one under 18?
Our national surveys typically interview the adult population both for its relevance to voting and political participation and practical difficulties in reaching respondents who are younger than that age (i.e. ethical considerations about obtaining parental permission to participate).
[My Response: This lack of representation should have been noted, as 50% of Native Americans are under the age of 29 and the data qualifed with that statement. Qualitative studies of Native American youth's feelings on the subject are available and could have been used to balance out the conclusions. In my article I note the White House Initiative on American Indian and Alaska Native Education report which contains testimony by hundreds of Native youth about the difficulties they face in school--including their feelings about being mascotted. 2,000 American high schools still mascot Native Americans.]
3) Why we're only 12% from mountain region where 18 out of 20 most populated Native reservations live?
The unweighted percentages of respondents by region are a function of the share of respondents who identified their race as Native American. Because of the geographic concentration of Native Americans in certain regions, the Post survey was weighted to match 2014 American Community Survey benchmarks for the population’s regional makeup. While the survey employed a weighting protocol designed to correct for relevant demographic and regional differences from the Native American population, the impact of weights on findings was minimal. In this case, respondents in the Mountain region were among the most likely to report enrollment with a tribe (67 percent did so), but the share who said the Washington Redskins’ team name was offensive was little different from the overall results (8 percent).
4) Why no mention of James Fenelon's survey, after he says he was contacted by reporters and asked if his was biased because he had Native Americans doing poll?
Dr. Fenelon’s survey was not based on a systematic national or regional sample of any population, but among a sample of attendees at selected pow wows and related events in the Cleveland area, as reported by Indian Country Today. We cannot use results from an unsystematic sample to make generalizations about the Native American population at-large.
5) how do you respond to NAJA statement? " “By framing this story as simply a matter of public opinion,” the NAJA/UNITY statement says, “the Post has willfully ignored the harm – referenced by the APA – that will inevitably result from its coverage. The reporters and editors behind this story must have known that it would be used as justification for the continued use of these harmful, racist mascots. They were either willfully malicious or dangerously naïve in the process and reporting used in this story, and neither is acceptable from any journalistic institution.”
The Post pursued this poll without any idea as to how it would turn out and had no vested interest in the outcome. When activists argue that Native Americans are offended by the name – and when debate over the name is at the center of a major public policy debate -- it’s entirely appropriate for a news organization to conduct a survey to test any assertions made about the breadth and depth of offense among Native Americans. This is customary for any other public policy issue.
From: Dave Z.
Sent: Tuesday, May 24, 2016 11:17 AM
To: Clement, Scott
Cc: Jacqueline Keeler
Subject: Questions
Hello Scott. Thank you for agreeing to respond to Jackie's article.
The questions are as follows. We are on deadline. Please get back today if at all possible. Jackie, please add questions if I am missing anything.
1) Why no verification of tribal enrollment, just taking people at their word that they are Native American?
2) Given how young the Native American community is, why speak to no one under 18?
3) Why we're only 12% from mountain region where 18 out of 20 most populated Native reservations live?
4) Why no mention of James Fenelon's survey, after he says he was contacted by reporters and asked if his was biased because he had Native Americans doing poll?
5) how do you respond to NAJA statement? " “By framing this story as simply a matter of public opinion,” the NAJA/UNITY statement says, “the Post has willfully ignored the harm – referenced by the APA – that will inevitably result from its coverage. The reporters and editors behind this story must have known that it would be used as justification for the continued use of these harmful, racist mascots. They were either willfully malicious or dangerously naïve in the process and reporting used in this story, and neither is acceptable from any journalistic institution.”
