RAND Corporation gun policy study

[Frank Ettin]

....My major concern would be that RAND isn't doing the studies themselves but just "validating" existing studies....

I've seen the use of these sorts of meta-studies in a number of contexts and dealing with various different subjects. Apparently it's a regularly used and accepted tool.

I think part of the issue is that these sorts of studies have certainly utility, but they also have limitations. This seems to be understood by professionals actually in the field.

So, for example, when I worked with clients in the medical field and they reviewed both clinical studies and meta-studies of aggregated clinical studies, they had a pretty good idea of what conclusions could reasonably be drawn and where questions remained.

The risks with any study, including a meta-study, is that folks who are not well schooled in both the subject matter and the nature of the analytical tools will misread and misapply the data. Confirmation bias continually rears its ugly head, and zealous advocates will find snippets that support their particular prejudices.

 

[Evan Thomas]

The risks with any study, including a meta-study, is that folks who are not well schooled in both the subject matter and the nature of the analytical tools will misread and misapply the data. Confirmation bias continually rears its ugly head, and zealous advocates will find snippets that support their particular prejudices.

Just so, Frank. And while it's laudable that RAND is attempting to make its findings accessible to the general public, the graphic, "interactive" format may make it all too easy to extract those snippets and ignore the overall picture. Wading through text may not be many people's idea of fun, but it does at least require some level of reading comprehension to get at the bits a motivated reasoner wants to find, while making it harder to overlook the bits he doesn't -- and perhaps makes it more likely he'll learn something along the way.