May 3, 2013 by Tamara Piety
A great post over at The Situationist blog about a new book from Mahzarin Banaji and Anthony Greenwald called Blind Spot: The Hidden Biases of Good People which is an attempt to bring some of their work on the Implicit Associations Test (IAT) to a broader, general audience. This work is interesting and provocative and, as noted in the blog, often controversial. But I suspect the book will be very interesting to those legal academics who have either written about the IAT or issues of implicit bias or cognitive limitations generally, as well as to practitioners who may be coming up against this sort of evidence in court or who, conversely, would like to use it in court.
The Faculty Lounge blogger Michelle Meyer brought a fairly critical review of the book to may attention from the WSJ blog, Book Review: Blind Spot. After looking at the review I think I’d have to say it goes beyond simply offering a caution about importing the insights (if any) of the IAT, but is fairly hostile to this research. It offers several of what seem like valid criticisms and a few that don’t seem so valid to me.
The observations about the linguistic limitations, the crudeness of some of the measures and cautions about the associations which the authors make, are examples of some of the valid critiques, although they do not necessarily lead to the conclusion that this work does not help to some degree in trying to tease out the difference between lying and lying to ourselves – or, in the context of discrimination, between what we consciously tell ourselves and what we actually do, or what may be unconsciously informing some of the decisions we make. I think some of the critics may be asking that this research conclusively prove discrimination, rather than that it show evidence which is suggestive of it or supports an inference in that direction. This is a very different thing. No evidence is like to conclusively prove unconscious bias. Indeed, conclusive proof of most connections in the social sciences is unlikely to be forthcoming, moreover, that is not the test for admissibility for expert evidence in court. Observing that a particular approach does not answer every question you might have with respect to the problem it attempts to study does not mean the test is invalid any more than the fact that an MRI and a CAT scan offer different information means that either is invalid. Finally, terms like “discrimination” “bias,” etc. are all themselves extremely loaded in that they might suggest more to some readers than the authors intend.
On the other hand, Daubert, the case that provides the framework for the admissibility of expert testimony, does provide that expert testimony needs to meet some basic standards of reliability, etc. that if all these criticisms are well-founded, would probably lead to exclusion. So, facts like the authors’ own meta-analysis failing to account for the vast majority of the data is, if true, a significant omission from the book and one that it would be interesting to follow up on. Indeed, if you were planning to use such evidence in court it would be remiss of you not to follow up on it.
I think other observations in the review are less valid. Take this one: “But if a test gives results that are so far-fetched, it’s time to start questioning the validity of the test.” First, “far-fetched” is an assessment that assumes the very thing that the authors are trying to discover, whether it is indeed “far-fetched” that someone might harbor unconscious racial stereotypes, no matter what their ethnic background. The comment assumes that racial heritage or background is some sort of invariable predictor of attitudes. A casual glance around at the diversity of political opinions within various minority groups should dispel that idea. So the ethnicity of Malcolm Gladwell’s parents, while surely not irrelevant to the question of his attitudes towards race, is probably not dispositive of them.
Second, that some data diverges from what you expect to find is not necessarily a reason to question the test. Sometimes it is. That is always one possibility. But it also is possible that it is a reason to question your assumptions about what is or is not “far-fetched.” There are a great many things which are true even though they are counter-intuitive, for instance that certainty is strongly correlated with accuracy.
Moreover, in this, as in so many areas of research that are highly politically charged, there are some hints that there may be ideological divisions presented as methodological ones driving some of the critiques. There does seem to be a way in which normative disputes about the legitimacy of underlying assumptions sometimes morph into arguments that purport to be about rigor or methodology, with one side claiming that the other is not really rigorous, when they may be working from radically different foundational assumptions. Whether this is going on here is a question that can only be resolved by further reading of the works on which the book is based and those critiques which the WSJ article references. I do not know the answer to that question. Interested parties should definitely read it all and decide for themselves.
At the end of the day, this book is, as advertised, a presentation of the authors’ research for a general audience, with all the limitations that go along with such attempts. That said, you need not endorse immediate adoption, by the courts or legislatures, of any of the implications of the authors’ research here to conclude that it is interesting and worth further exploration. And, for better or worse, as the National Research Council’s report from 2009 on forensic science illustrates, the validity of a practice has rarely been an insurmountable barrier to its acceptance in the courts. So this research is likely to migrate there. (I think it may have done so already and look to more knowledgeable readers to alert me to those cases.) Of course politics and power play a role over how readily it may be adopted. As we see from the example of forensic sciences, a great deal of flimsy research and unsupported assumptions have been permitted for years in criminal cases. That this sort of testimony tends to confirm existing biases and is usually offered by the prosecution may have something to do with the evidentiary double standard. Whether you want to use IAT or defend against it, it may pay to give this book a look.
A version of this post first appeared on The Faculty Lounge here