Call It A Comeback
I remember an evening about 10 years ago when there were competing rallies concerning an issue affecting lesbians. I was in awe of the fact that there were enough politically active lesbians to pull off dueling protests. Madison is very unlike the rest of the Midwest and this story is only an example.
Normally, I’m most disturbed by the conflation of quotas and affirmative action, but lately I’ve been just as disturbed by college administrators who use the term diversity to justify affirmative action. Diversity is important and should be sought out, but more important to the underlying need for affirmative action is the need to promote opportunity. Nancy Cantor, current U of Illinois Champaign Chancellor argues for some race based factors in admissions, but fails to mention the importance of opportunity for underrepresented minorities.
This woman couldn’t be more screwed up.
Michael Sneed reports:
Sneed hears rumbles indictments of some McCormick Place honchos may be imminent. Hmmmm. Does it involve McPier Board contracts?
Gulp!
This will be ugly.
U.S. Senate hopeful Barack Obama, who was ecstatic when former U.S. Sen. Carol Moseley-Braun pulled out of her bid to recapture her old seat, may still have to hit the bottle of Excedrin.
*The reason: Former lieutenant governor hopeful Joyce Washington is considering entering the Senate fray.
*The extra reason: Moseley-Braun’s exit meant Obama could wake up with at least a guaranteed 32 percent of the Dem primary vote. (Obama, a state senator, was just voted among the top 100 Dem leaders to watch by the national Democratic Leadership Council.
DLC doesn’t have this posted yet. Joyce Washington? One has to wonder if she is a real candidate.
Judy. Judy. Judy. Governor Judy?
Sneed hears the White House views state Treasurer Judy Baar Topinka, the only Republican to win statewide election, as a formidable candidate for Illinois governor in 2006.
*To wit: Isn’t that one reason Topinka is heading to Washington, D.C., this week, where she will be queried about spearheading the Bush re-election campaign in Illinois? You bet.
Oh, how the conservatives will howl…
Over at the Political State Report.
Firing of Ryan Appointees and the Favors List.
Speaking of the Favors list, Mark Brown does an excellent job of providing context and making one laugh.
It is definitely going to be money versus organization in the Democratic Primary, the Sun-Times reports Hull is willing to spend $40 million of his own money.
If he doesn’t get Mell/Blagojevich backing this looks like the second grounding of Al Hofeld minus the later personal problems.
Washington Whispers is reporting that Hizzoner is considering a run at Fitzgerald. I don’t buy it. Running Chicago is his thing and there are no natural torch bearers to hand it to right now. My guess is US News got it confused with Daley actively seeking a strong challenger to Fitzgerald.
The question John Lott won’t answer is why do a survey incapable of providing valid and reliable results for the question he is trying to ask. Lott has been shopping around the number that 98% of DGU (Defensive Gun Uses) occur without a shot being fired. This is significantly different than the results of others studies. No other study shows a figure of less than 21% of DGUs involving the weapon being fired. I’ve asked him three times and he has yet to provide a substantive response.
If one thinks there is a major methodological flaw that is inflating the rate of DGUs with a discharge, one would design a study that improved upon previous designs. His does not. It offers an instrument that is rudimentary, sampling methods that are suspect to be generous, and a sample size smaller than at least one previous study. In addition, the execution of both the first survey and second survey can kindly be called amateurish. The question I have asked him three times is how does his study hope to improve upon the 1995 paper by Kleck and Gertz. His only on topic response was:
There are significant problems with using a five year window. Sure it helps you get a bigger sample of defensive gun uses, but there is also a lot more error. For example, using five years is likely to results in respondents including cases that go back even further than five years. Answers to questions about what happened are also likely to contain more errors.
This is a terribly misleading response given that Kleck and Gertz’s 1995 piece looked at both 5 year and 1 year time frames because of the issues involved in 5 year recollections. In the single year time period, the Kleck and Gertz survey recorded 56 respondent DGUs and 68 Household DGUs, both numbers over twice the number in Lott’s first survey. Now in both surveys these samples are too small to make too strong of claims regarding the subsamples of DGUs. However, given Lott had 25 DGUs, his MOE is +/- 20 percent compared to Kleck and Gertz with +/- 13.4 percent (using individuals since Lott didn’t check on households–this is also problematic–see Lamber from the 12th). Even if Lott is foolish enough to try and claim both surveys as his sample size, he would end up with fewer cases than Kleck and Gertz. The newer survey being conducted has provided about 1015 responses and so if it is truly consistent with the mysterious first survey, the responses would only produce 10.5 more DGUs for a total of 35.5. This is still lower than the K & G survey sample of one year reports. And I made a mistake yesterday. Tim Lambert found the 1 year time frame produced identical rates for firing of the 5 year time frame respondents. Mea Culpa. As a note, Tim has been on this issue for far longer and while I’m hoping that I’m adding something, he has a far greater amount of information.
His other responses were complaining about what he feels is a misrepresenation by me:
As for the 2002 survey, a number of calls (form the surveyors end) were indeed randomly listened to by me. In all defensive gun uses, the surveyors were debriefed that night or the following morning about the call. All the respondents in these cases volunteered extensive details of what happened with the defensive gun use. None of the defensive gun uses recorded involved defensive uses by police. A couple of our surveyors had previous experience and I asked them to talk to the other surveyors before surveying began. As a result of call backs, over 50 percent of telephone numbers produced completed interviews.
Given the picture emerging of John Lott’s understanding of survey research, I’ll stand by my comment that the second survey isn’t an improvement and there was no effective supervision of callers. He does claim that a couple of the callers had some experience and they talked to the others. My, how comforting. Apparently he feels that his surveys are of the quality of the Kleck and Gertz survey, but can’t explain why.
John Lott is apparently uninterested in explaining how his survey is an improvement of better executed surveys in the past. Given this, one has to question whether Lott has a goal besides covering himself. Neither survey would be an improvement upon the Kleck and Gertz work or other works and so one is left with the uncomfortable feeling that John Lott is in the business of producing results that fit what he wants and not what he observes. The alternative is that he doesn’t understand the extent of the problem. Ignorance is not a flattering excuse in this case.
But to take this one step further, while Kleck and Gertz are limited in their one year conclusions, their survey is inadequate to make strong conclusions from and they point this out in their article. To add to the literature, Lott would need to not just meet that survey’s quality, he would need to improve upon it with a higher number of DGU respondents. Instead, he will have fewer, but he expects the public to accept the results. That’s chutzpah.
Jacob Levy added a comment regarding the incident. The IRB issues has been dismissed because the Chicago Law School apparently ignored such rules. This is disturbing, but arguably Lott was a part of a larger problem in that case. What is most disturbing, as I’ve said before, is that Lott had survey research with connecting information being stored and entered in dorm rooms. This isn’t a technical violation of the treatment of human subjects, this is a serious violation of how such material should be stored to protect human subjects’ privacy. Of all people, gun rights advocates should understand the importance of this.
In a larger sense, my views on gun control have largely been shaped by works like that of Kleck and Gertz. I don’t necessarily have a problem with concealed carry, I dislike HCI for many of the same reasons I dislike Lott, and I think guns should be generally available to the public though preferably with an FOID system as Illinois has. Why would some gun rights advocates work so hard to protect a charlatan when other credible researchers are out there that make solid arguments concerning the defensive use of guns that are overwhelmingly positive for gun rights advocates? Clearly, not all gun rights advocates take Lott seriously, but it seems that he is considered by some advocates as more credible and impressive than people like Kleck. I don’t get it.
Rich Miller at the Capitol Fax reports:
"GREEN" ENERGY BILL COMING Crain’s Chicago Business reported over the weekend that a bill will be introduced soon to require the state’s electric utilities to purchase a portion of their power from renewable sources.
The bill, which will be sponsored by Sen. Pat Welch (D-Peru), will force electric companies to buy 5 percent of their power from renewable sources by 2010, and 15 pecent by 2020. The utilities question the reliability and availability of alternative sources like wind power and say it costs more. But proponents claim the cost is about the same for wind and natural gas (3 cents per kilowatt-hour for gas and 3 to 4 cents for wind), and say the initial 5 percent requirement is sufficiently modest. Several companies are hoping to build "wind farms" in central Illinois, according to the article, but the investments may not be made without the legislation.
This is great news. One of the critical aspects of encouraging green energy is creating a market for it. This would provide for competition between such suppliers and in the short description, ultimately make it affordable.
From John Lott:
Here is a suggestion. If you have a question about whether something was done in the survey, it might make more sense to ask the question whether it was done rather than asserting that it must not be true. I had simply asked James to write up specific points, particularly how the survey sample was gathered. The discussion that I sent you dealt with the issues that had previously been raised.
There are significant problems with using a five year window. Sure it helps you get a bigger sample of defensive gun uses, but there is also a lot more error. For example, using five years is likely to results in respondents including cases that go back even further than five years. Answers to questions about what happened are also likely to contain more errors.
So Lott wants to claim that his survey ‘improves upon’ Kleck and Gertz (1995) because he limits it to one year. That might have some merit if he used the same or better survey techniques and his sampling was anything close to the quality of Kleck and Gertz’s.
He didn’t. In fact, the Kleck and Gertz survey called back to verify results on all DGUs, instituted several criteria to ensure accuracy in relation to actual civilian defensive use, and significant screening was used to determine the nature of the reported DGU unit.
Anytime anyone is asked to report a memory, the farther back the event, the lest trustworthy the memory. Lott is correct in this and this is why one would use probe the memory and attempt to establish credibility. Additionally, self-reporting can often inflate the reports, something Kleck was very concerned with, something Lott, despite protestations of being provided extensive details by all DGU respondents, did not do in the instrument.
Of course, what Lott is not mentioning and is highly relevant, is that Kleck and Gertz asked about DGUs within the last year as well. In fact, they have more cases of that subsample than does Lott per Table 2 (scroll down). Kleck and Gertz don’t break down the one year rate of a weapon being fired, but it is reported as being higher than in the 5 year reports. Of course, the sample size is small and has a larger maring of error than the 222 total (or 213 depending on which sample they are discussing). That MOE is still smaller than Lott’s and it is from a far better constructed survey.
For a good discussion of the issues surrounding 1 or 5 year samples see Kleck and Gertz. Lott’s complaint about the 5 year window is especially curious in this case given Kleck and Gertz explicitly cover the issue in detail and account for it.
As for the 2002 survey, a number of calls (form the surveyors end) were indeed randomly listened to by me. In all defensive gun uses, the surveyors were debriefed that night or the following morning about the call. All the respondents in these cases volunteered extensive details of what happened with the defensive gun use. None of the defensive gun uses recorded involved defensive uses by police. A couple of our surveyors had previous experience and I asked them to talk to the other surveyors before surveying began. As a result of call backs, over 50 percent of telephone numbers produced completed interviews.
In a pattern that is becoming all too apparent, Lott tries to equate a fly by the seat of your pants approach to doing his survey to having trained surveyors.
This speaks to Lott’s poor understanding of surveys. Even individuals who do a lot of survey research generally rely upon survey experts to conduct and help construct the surveys. The relevant passages in Kleck and Gertz are:
The present survey is the first survey ever devoted to the subject of armed self-defense. It was carefully designed to correct all of the known correctable or avoidable flaws of previous surveys which critics have identified. We use the most anonymous possible national survey format, the anonymous random digit dialed telephone survey. We did not know the identities of those who were interviewed, and made this fact clear to the Rs. We interviewed a large nationally representative sample covering all adults, age eighteen and over, in the lower forty-eight states and living in households with telephones. [42] We asked DGU questions of all Rs in our sample, asking them separately about both their own DGU experiences and those of other members of their households. We used both a five year recall period and a one year recall period. We inquired about uses of both handguns and other types of guns, and excluded occupational uses of guns and uses against animals. Finally, we asked a long series of detailed questions designed to establish exactly what Rs did with their guns; for example, if they had confronted other humans, and how had each DGU connected to a specific crime or crimes.
We consulted with North America’s most experienced experts on gun-related surveys, David Bordua, James Wright, and Gary Mauser, along with survey expert Seymour Sudman, in order to craft a state-of-the-art survey instrument designed specifically to establish the frequency and nature of DGUs. [43] A professional telephone polling firm, [Page 161] Research Network of Tallahassee, Florida, carried out the sampling and interviewing. Only the firm’s most experienced interviewers, who are listed in the acknowledgements, were used on the project. Interviews were monitored at random by survey supervisors. All interviews in which an alleged DGU was reported by the R were validated by supervisors with call-backs, along with a 20% random sample of all other interviews. Of all eligible residential telephone numbers called where a person rather than an answering machine answered, 61% resulted in a completed interview. Interviewing was carried out from February through April of 1993.
The quality of sampling procedures was well above the level common in national surveys. Our sample was not only large and nationally representative, but it was also stratified by state. That is, forty-eight independent samples of residential telephone numbers were drawn, one from each of the lower forty- eight states, providing forty-eight independent, albeit often small, state samples. Given the nature of randomly generated samples of telephone numbers, there was no clustering of cases or multistage sampling as there is in the NCVS; [44] consequently, there was no inflation of sampling error due to such procedures. To gain a larger raw number of sample DGU cases, we oversampled in the south and west regions, where previous surveys have indicated gun ownership is higher. [45] We also oversampled within contacted households for males, who are more likely to own guns and to be victims of crimes in which victims might use guns defensively. [46] Data were later weighted to adjust for oversampling.
Each interview began with a few general "throat-clearing" questions about problems facing the R’s community and crime. The interviewers then asked the following question: "Within the past five years, have you yourself or another member of your household used a gun, even if it was not fired, for self-protection or for the protection of property at home, work, or elsewhere? Please do not include military service, police work, or work as a security guard." Rs who answered "yes" were then asked: "Was this to protect against an animal or a person?" Rs who reported a DGU against a person were asked: "How many incidents involving defensive uses of guns against persons happened to members of your household in the past five years?" and "Did this incident [any of these incidents] happen in the past twelve [Page 162] months?" At this point, Rs were asked "Was it you who used a gun defensively, or did someone else in your household do this?"
All Rs reporting a DGU were asked a long, detailed series of questions establishing exactly what happened in the DGU incident. Rs who reported having experienced more than one DGU in the previous five years were asked about their most recent experience. When the original R was the one who had used a gun defensively, as was usually the case, interviewers obtained his or her firsthand account of the event. When the original R indicated that some other member of the household was the one who had the experience, interviewers made every effort to speak directly to the involved person, either speaking to that person immediately or obtaining times and dates to call back. Up to three call- backs were made to contact the DGU-involved person. We anticipated that it would sometimes prove impossible to make contact with these persons, so interviewers were instructed to always obtain a proxy account of the DGU from the original R, on the assumption that a proxy account would be better than none at all. It was rarely necessary to rely on these proxy accounts– only six sample cases of DGUs were reported through proxies, out of a total of 222 sample cases.
While all Rs reporting a DGU were given the full interview, only a one-third random sample of Rs not reporting a DGU were interviewed. The rest were simply thanked for their help. This procedure helped keep interviewing costs down. In the end, there were 222 completed interviews with Rs reporting DGUs, another 1,610 Rs not reporting a DGU but going through the full interview by answering questions other than those pertaining to details of the DGUs. There were a total of 1,832 cases with the full interview. An additional 3,145 Rs answered only enough questions to establish that no one in their household had experienced a DGU against a human in the previous five years (unweighted totals). These procedures effectively undersampled for non-DGU Rs or, equivalently, oversampled for DGU-involved Rs. Data were also weighted to account for this oversampling.
Questions about the details of DGU incidents permitted us to establish whether a given DGU met all of the following qualifications for an incident to be treated as a genuine DGU: (1) the incident involved defensive action against a human rather than an animal, but not in connection with police, military, or security guard duties; (2) the incident involved actual contact with a person, rather than merely investigating suspicious circumstances, etc.; (3) the defender could state a specific crime which he thought was being committed at the time of the incident; (4) the gun was actually used in some way–at a minimum it had to be used as part of a threat against a person, either by [Page 163] verbally referring to the gun (e.g., "get away–I’ve got a gun") or by pointing it at an adversary. We made no effort to assess either the lawfulness or morality of the Rs’ defensive actions.
An additional step was taken to minimize the possibility of DGU frequency being overstated. The senior author went through interview sheets on every one of the interviews in which a DGU was reported, looking for any indication that the incident might not be genuine. A case would be coded as questionable if even just one of four problems appeared: (1) it was not clear whether the R actually confronted any adversary he saw; (2) the R was a police officer, member of the military or a security guard, and thus might have been reporting, despite instructions, an incident which occurred as part of his occupational duties; (3) the interviewer did not properly record exactly what the R had done with the gun, so it was possible that he had not used it in any meaningful way; or (4) the R did not state or the interviewer did not record a specific crime that the R thought was being committed against him at the time of the incident. There were a total of twenty-six cases where at least one of these problematic indications was present. It should be emphasized that we do not know that these cases were not genuine DGUs; we only mean to indicate that we do not have as high a degree of confidence on the matter as with the rest of the cases designated as DGUs. Estimates using all of the DGU cases are labelled herein as "A" estimates, while the more conservative estimates based only on cases devoid of any problematic indications are labelled "B" estimates.
The question remains, what does John Lott think his survey is going to produce of value? Comparing the two methods of conducting a quality survey identifies how unconcerned Lott is with his research’s accuracy.
ArchPundit