Navigation
Search
New Comments
Handy Links
Syndication
« A Primer on Testing Issues | Main | a/s/l/research? »
Internal Stakeholders' Reactions to Testing
On October 31, 2008 by Jamie Madigan
Okay, if there are any grad students out there looking for a thesis/dissertation idea, I'm going to give you a freebie here. In the recent issue of SIOP's new journal, Organizational Psychology: Perspectives and Practice Scott Highhouse has a nifty little article entitled "Stubborn Reliance on Intuition and Subjectivity in Employee Selection." This article could easily also been titled "What? No! Why Are the Hiring Managers Doing That? Make Them Stop!" because it basically looks at what he sees as two of the root causes for organizational decision-makers to reject or circumvent scientifically derived selection systems like employment testing.
First is the belief that it should be possible to explain 100% (or close to it) of the variance in human behavior within an organizational context. Someone holding this belief may scoff at your puny validities -- 11% of variance explained? Pshaw! Humans are just squishy machines, right? We should be able to predict their performance perfectly. Your expectancy tables and realistic discussions of false positives are powerless in the face of this belief.
The second common reason for objecting to selection systems is the belief that experience makes people better at figuring someone out. This comes through intuition, hunches, reading between the lines, and other nebulous decision-making. Your test results may not mean much if the interviewers like the cut of the candidate's gib. Just the fact that the guy actually brought in a gib to show them how he had cut it won them over.
Anyway, Highhouse's article discusses the origins of these troublesome beliefs, and several of the follow-up articles in the same article discuss how to combat them. This, though, made me realize that there's a whole nascent line of research that's just waiting to be expanded: stakeholder reactions to selection systems.
Think about it. There's a great and thriving body of research on applicant reactions to testing, drug screens, and other selection systems. I should know --it was the topic of my Ph.D. dissertation. We know how to study this kind of thing, so why hasn't anyone turned their attention to building and testing theories about the reactions of other stakeholders, like hiring managers and other decision makers?
We could do it. Heck, you could use the applicant reactions literature as a template to get started. Do hiring managers dislike aptitude tests because of their lack of face validity or because they rob them of control over the decision making process? What kinds of biases and kinks of the human mind come into play when trying to understand probability and utility in a selectoin testing context? Are hiring managers more likely to support testing if they get to interview candidates before or after testing? If we knew more about what kinds of test characteristics drive what kinds of reactions among internal stakeholders, we testing professionals would be better equipped to address, assuage, and prevent those concerns without sacrificing the validity and utility of our tools.
So, there you go. Somebody get on that. If I can find time, I certainly will.
Existing comments:
Post your own comment:
