a weblog covering personnel selection, assessment, and pre-employment testing in the news. updated fridays

The Importance of a Good Shake

On May 8, 2008 by Jamie Madigan

Handshake

What's the best way to ace an interview? Come prepared with facts and examples of how your qualifications make you a strong fit with the requirements of the position? Nope. Apparently you just need a firm handshake. The kind that says "I have a high composite score on measures of extroversion and emotional stability AND I WILL CRUSH YOU!"

Well, that's the story from this article, which describes some research forthcoming in --of all places-- The Journal of Applied Psychology. The research, conducted by professor Greg Steward at the University of Iowa, describes an experiment where student subjects did mock job interviews and had their handshakes systematically evaluated. Those with beefier grips had higher ratings of their interview performance.

Now, we've all heard or read little bits of advice about how to dress, how to speak, and how to shake hands at an interview, but unless the person is interviewing for the job of Bare-Fisted Walnut Crusher (O*Net Online job code #39-2011.00, look it up), it's a little surprising that handshakes would have a particularly strong effect on objective interview ratings.

In fact, I look forward to reading the study when it comes out in JAP. I'm not sure if this is a case of the author of the article linked above selectively quoting to make a more interesting story, but this bit by the researcher made me arch an eyebrow:

"We probably don't consciously remember a person's handshake or whether it was good or bad," Stewart said. "But the handshake is one of the first nonverbal clues we get about the person's overall personality, and that impression is what we remember."

Of course there's tons of research on interview biases and first impressions, but one would hope that if you wanted to measure someone's personality, you'd choose a better way to go about it. There ARE tests for that kind of thing in wide use. Again, I look forward to learning more about the research design, but what this really points to is the need for more structured interviews that were designed according to well established job analyses so that not only do you have interview content that's job related, but you've got tools and procedures built into the interview process that minimize biases born of handshakes or other silliness. It would be interesting, for example, to study whether or not the degree to which the interview was structured moderated or mediated the handshake/interview performance relationship.

Still, if you want my advice on handshakes, here's how to pull one off that's sure to make an impression: Grasp the other person's hand as hard as you can. If they whimper and maybe bend at the knees, you're doing it right. Lean forward until you make eye contact --as in, your eyeballs actually make contact with the other person. Pump the hand up and down twice, but don't let go. Seriously, don't let go until the interview is over even if it means you have to do the whole thing with your arm stretched out across the table to the other person. If there is a second interviewer, use your free hand to repeat. If there are three or more interviewers, bring a friend.

Let me know how this works out for you. I'm curious to hear.

  Comments (1) |   Permalink 


Integrity issues at the TSA?

On April 23, 2008 by Jamie Madigan

TSA

Though it's light on actual facts besides a few claims by irate travelers and a blanket statement from the accused, this article on MSNBC's website claims that thieving in the Transportation Security Administration (TSA) is rampant. The TSA is the organization that employs baggage checkers and those people who make you take off your belt and shoes at airport security checkpoints.

Though the article only digs a bit to mention some stats to support this claim (such as firing about 200 employees for theft since 2001), it is interesting because this is the kind of thing you usually hear about in retail operations. One has to wonder if integrity testing would be a useful addition to whatever other kinds of screening the TSA does on its applicants, assuming they don't use one already.

  Comments (2) |   Permalink 


Best Buy HR Manager: "Don't hire him, he sucks"

On January 30, 2008 by Jamie Madigan

Best Buy

This is an interesting story about how NOT to do reference checks. When a former Manager at superretailer Best Buy was fired, he went on to apply at a few big box stores. When he abruptly ran up against the stone wall of rejection after a couple of promising starts, he got suspicious and did a little sleuthing. He faked an e-mail from a Target employee and e-mailed his former employer asking for a reference.

According to the lawsuit that followed these shenanigans, the Best Buy District Human Resources Manager allegedly replied thusly:

"I will give you the skinny on him but you can't say you got any info from Best Buy or we can be sued. Just don't hire him and say you went with a better candidate.

"He was hired as GM and demoted after 12 months or so because he sucked. He is desperate for a job because supposedly his wife left him because he has no job. I would not touch him.

"Again, do not forward this e-mail to anybody or say where you heard the info from because we were not allowed to give this info out, but I would hate you to get stuck with this guy!"

Oops, if true.I think there are two lessons here. One, be careful what you put in e-mails. You never know for sure who you're talking to, nor to whom the e-mail will be forwarded.

More importantly, though, the second lesson is that if you give out references on ex-employees, be objective. How much information you give out is up to you and how much legal exposure you can tolerate, but in general keep it objective, quantifiable, and try to avoid the use of the word "suck."

Of course, I'm not a fan of personal references at all. The research I've seen (a couple of articles in refereed journals and a SIOP presentation or two) is pretty clear that they're worthless. There's no variance (i.e., this person at Best Buy aside, almost all references provide nothing but unqualified praise), and they don't predict performance worth a darn.

  Comments (0) |   Permalink 


Practitioner Periodicals vs. Academics: Round One, FIGHT!

On January 24, 2008 by Jamie Madigan

There's a great article in a recent issue of Academy of Management Journal titled "The Very Separate Worlds of Academic and Practitioner Periodicals in Human Resource Management: Implications for Evidence-Based Management." It's a mouthful, but basically what the researchers did was pick certain facts/issues about which academics feel mighty confident and then examine how those issues were covered by three more mainstream Human Resource Management publications:

  • HR Magazine (the publication of the Society for HR Management, or SHRM)
  • Human Resource Management (a "bridge" publication that theoretically appeals to both practitioners and researchers)
  • Harvard Buisiness Review (which admittedly isn't HR specific, but which is definitely read by a lot of HR managers)

The researchers started with a medium-sized list of findings from the academic research that they felt (probably rightfully so) that are de facto true at this point, but pared it down to three:

  1. The importance of intelligence or general mental ability for job performance
  2. The importance of goal setting and feedback for job performance
  3. The validity of personality measures (including integrity tests) as predictors of job performance

With two of those factors directly related to many selection tests, you can see why this piqued my interest. The researchers wanted to know how much these topics were covered, and when they were covered how in line with academic knowledge they were.

The long and the short of it is that these publications VERY RARELY mentioned any of these topics --at most, just over 1% of the articles mentioned anything related to any of the three. HR Magazine --the official publication of the largest professional society for HR Professionals in the world and with a circulation of over 200,000-- didn't mention cognitive ability testing AT ALL and barely mentioned personality or goal setting.

Let's think about that: If the researchers had also included in their sample The Archie and Jughead Double Digest and the assembly instructions that came with that end table you bought from Ikea, HR Magazine would not have scored any better than these.

The few times that these three periodicals did mention cognitive ability, personality, or goal setting, what they said was sometimes in line with academic research, but it was also often wrong, incomplete, or just bizarre. My favorite was from one of the very few articles on personality testing that recommended not that you use a scientifically developed and scrupulously validated personality test to screen applicants, but rather suggested this:

You can pick up a multitude of clues about a person's character by simply having a restaurant meal together. You'll see how they interact with the waiter or the people sitting at adjacent tables. I sometimes say, "Gee, how much tip do you think we should leave?" Then, based on whatever percentage they suggest, I ask why. I want to see how they make those decisions. A lot of it bears on how they view the world in a more general sense.

The authors of the Academy of Management article say, quite diplomatically, that "This quote represents a selection tactic that is low in validity and utility but high in exposure to potential legal liability." Personally, it reminds me of the time I interviewed for a job and later found out that one interviewer had dinged me because I had somehow chosen the wrong chair to seat myself in.

The article goes on, describing the different things they found in this audit of the 3 journals, and by the end the finding is clear: mainstream journals don't cover this stuff very frequently (if at all), and when they do they often get it wrong. They go on to explain some reasons for this and some ways to fix it, which I'd encourage you find out about if you have access to the AMJ article. There are also other articles that respond to this study and the topic in general, but I haven't had a chance to read them yet.

I think a good follow-up to this kind of research would be to do an audit of not HR magazines, but general HR textbooks. If education is the problem, this seems like a good place to start and an easier problem to fix.

  Comments (3) |   Permalink 


My Column in TIP

On January 16, 2008 by Jamie Madigan

TIP

The column that I co-author in The Industrial Psychologist is now up. Actually, it's been up for a while, I've just forgotten to mention it.

In this issue, my part of the column focuses on a series of articles in a recent Academy of Management Journal that themselves focus on the Scientist/Practitioner divide. There's some interesting stuff there, and I recommend the series of articles if you have easy access to AMJ (it's from volume 50, which I think came out in late in 2007). A lot of pretty smart people took a stab at not only solutions to the scientist/practitioner problem, but at speculation about whether or not the divide actually exists in the first place, as well. One of the repeated themes is "teach, teach, use your own research to teach your students and your executive education clients). This makes a lot of sense to me. One of the authors made the very good point that academics should seek to open their loop, so to speak, by seeking out audiences and collegues outside of their normal circles. Look, don't just show your tidy little correlation matrix to your research assistants and your department chair --figure out how to get your findings into the heads of more people running businesses or other organizations. It's not easy and it's most definitely NOT routinely encouraged by the academic reward systems ("publish or perish," as the saying goes), but it's more in line with the mission of I/O Psychology than a lot of things you could do.

Anyway, read the whole column here. And as a side note, this is the first column where I finally got rid of that awful, awful photograph of me that accompanies each publication of the column. It was a full-on snapshot hastily taken with a cheap digital camera, and though I always thought I looked like some goofball instead of someone who actually knows what he's talking about I never seemed to get around to replacing it until now. The reason I was able to is that I had taken a photography class (a hobby of mine) and in one session we worked in a studio with proper studio lighting. When they asked for a volunteer I jumped at it, stipulating that I'd like a copy of the pics.

So, big improvement. I did, however, briefly consider going with this version of the portrait, which resulted when the front flashes failed to fire:

Mystery Date


It's got kind of a "Mystery Date" vibe going, no?

  Comments (0) |   Permalink 


EEOC Issues Fancy New Fact Sheet

On January 4, 2008 by Jamie Madigan

EEOC

Capitalizing, perhaps oddly, on the Harry Potter mania, the U.S. Equal Employment Opportunity Commision (EEOC) has released a new "fact sheet" on employment testing and selection practices.

Not to give out any spoilers, but the information is pretty basic and straight forward: what selection tests are, what the relevant laws are, a smattering of recent rulings, and links to more information. This is a handy link to have in your favorites browser, as I could see it as useful for forwarding on to managers who need a refresher or overview of this kind of thing.

  Comments (0) |   Permalink 


The Current State of Integrity Testing

On December 21, 2007 by Jamie Madigan

integrity

There's a pretty good article in a recent issue of Personnel Psychology by Christopher Berry, Paul Sackett, and Shelly Weismann called "A Review of Recent Developments in Integrity Test Research." It's a readable, relatively non technical read that you may want to check out if you use or think you may use integrity tests. Or if you want the short and skinny of it, there's currently a write-up on the front page of SIOP.org. But I can't figure out how to direct link to it so it may be gone by the time you read this.

You know integrity tests. They're those sneaky paper and pencil tests that have questions like this:

Q1 - Have you ever stolen anything?
a. What? No way, man.
b. Nuh-uh.
c. Nope, never, nope.
d. Why, I never! Look, you've made me drop my monacle!

Q2 - No, seriously dude, have you ever stolen anything?
a. Oh, sure. All the time.
b. Yeah. I'm stealing this pencil right now!
c. No. Never. But tell me: Where do you keep the petty cash?
d. Well, just some office supplies. And millions in pension benefits.

Well, maybe not quite like that unless we're talking about the world's worst overt integrity test. The Berry, Sackett, and Weismann article gives you a much clearer picture of where things stand with integrity testing. Perhaps the section most fascinating to me was the one dealing with how item-level analysis of integrity tests revealed that they may be tapping something outside of the Big 5 personality taxonomy. Because anything that stretches us beyond that holy pentagram of personality research is potentially a good thing.

And integrity tests do appear to work, despite all the grumbling about them and people who say that folks will just lie when completing them. I've talked to a few people about this kind of thing, and beyond the typical biases surrounding personality testing there just seems to be this miasma surrounding integrity testing in particular. Maybe they just don't want applicants to think they're being treated like criminals, or maybe people just refuse to believe the test validities aren't affected much by faking (which they typically aren't.). Who knows. But it's very practical article that should probably be kept on hand for when it's needed.

  Comments (0) |   Permalink 


A Failure to Communicate

On December 14, 2007 by Jamie Madigan

Here, I want to share two quotes with you, neither of which make much sense to me. First, from the newsletter of a major management consulting firm:

Enterprise risk management is the proactive execution of a senior-management sponsored, entity-wide assessment and response to the collective risks that impact an organization's ability to maximize stake holder value. While many acknowledge that developing a leadership pool is critical for business, few see the lack of leadership or lack of succession planning as a business risk.

What? Huh? I can barely make it past the first sentence the communication barrier is so high. It's like it's being spoken by some guy on the Moon who's speaking French for some reason. And I don't speak French. I only speak a little Spanish, so my standing there and shouting "¿¡Adonde esta la biblioteca!?" over and over again isn't going to get us anywhere. Seriously, I have no idea what it is that the author is talking about there. It's business speak gone out of control.

But wait, let's look at something from the scientist part of the scientist-practitioner spectrum. Here's an excerpt from something published in the latest Journal of Applied Psychology:

Table 3 shows the correlations and descriptive statistics for the study variables. There were positive correlations between (a) supervisor-rated task performance and (b) extroversion (r = .15, p < .05) and positive moods (r = .16, p < .05). A preliminary step for HLM began with fitting a null model to estimate the total systematic variance in the dependent variable; this resulted in an intraclass correlation coefficient (ICCI) of .21.

Okay, so I happen to speak whatever language this is written in, but I imagine a lot of people who are interested in that study's general topic (the effects of being happy on job performance) don't. Or they only speak a little, so they could only haltingly ask "Please which way to the alligator station please?" in response. For most people, the message is equally ineffable.

Before I go on, I want to emphasize that I'm not singling any of these particular authors out. They're obviously very good at what they do. Better than me, certainly. And yes, the quotes are devoid of context which makes them seem more dense than they really are, but I got a point to make. And that point is that addressing the practitioner/scientist divide in I-O Psychology is a hot topic right now, and a lot of people are talking about how to make academic journals more accessible to lay people by using less esoteric language and communicating the research within a more realistic context. This is a good discussion to have, and I think that there's a lot of low hanging fruit to be captured if those who publish in the academic press can just unstraddle their preoccupation with certain styles and traditions. My part of the upcoming "Good Science Good Practice" column in TIP deals a lot with this.

But what really struck me about that first quote from the management consulting firm is that this argument can be applied to both sides of the spectrum. Granted this is partially marketing speak, but I've seen a lot of brochures, websites, and even white papers that use similar language at the expense of clarity and effectiveness. When I read stuff like that I have to either just ignore it or look for the contact information of a person I can actually talk to. Perhaps some of those crying for academics to change their tone should consider changing their own.

  Comments (2) |   Permalink 


I have the personality of a bunny

On December 7, 2007 by Jamie Madigan

The new movie The Golden Compass is set in a world where part of people's souls (called, perhaps oddly, "demons") live outside their body in animal form. The animal that one's demon imitates is supposed to say something about your personality --more subservient people may have dog demons, while scholarly ones may have owls, and Nichole Kidmans have golden monkeys.

To help promote the movie, the official website has set up a quiz to help you determine what kind of demon YOU would have if you lived in this magical world. The site is in awful full flash animation, so I can't directly link to the quiz but if you go to the site and choose the "demons" tab you'll get to a page where you can choose "meet your demon."

The quiz consists of items that look an awful lot like what you might find on personality tests used in pre-employment testing:

At parties you prefer not to talk to people you don't know.

It doesn't take much to get you fired up.

You usually get your own way.

And so forth. Test-takers are asked to respond to each statement using a standard Likert-type scale of Strongly Disagree to Strongly Agree.

My Demon

I took the quiz and my demon is a bunny! And apparently bunnies are "modest, assertive, shy, passive, and responsible." I'm ...not quite sure what to make of that. The quiz is obviously more like a horoscope and less like any kind of scientific test, but I can't help wondering how it was put together and what kind of model it used. There are many different demons that you can get by taking the test different ways so some thought must have gone into it. I wonder if they consulted any kind of real expert? I'm guessing not, but it's interesting to speculate.

As a testing professional, I'm of two minds with something like this. On the one hand, it's fun and kind of cool to see that something like personality testing is getting mainstream enough to be used in this way. On the other hand, it's fairly disheartening to see it treated like a cheap marketing gimmick, and one that's associated with magic and parlor tricks. But then again, we bunnies tend to take things like that seriously.

  Comments (0) |   Permalink 


Link and Run #5 (7/20/07)

On July 20, 2007 by Jamie Madigan

Links for your clicking pleasure.

  Comments (0) |   Permalink 


My New Column is up in TIP

On July 13, 2007 by Jamie Madigan

TIP

Looks like the July 2007 edition of The Industrial-Organizational Psychologist (a.k.a., TIP, SIOP's quarterly magazine) is online, including my column that I co-author with Marcus Dickson. The column is "Good Science Good Practice" and in each issue we try to highlight research that bridges the gap between scientists and practitioners. Preferably without resorting to cage matches. This time we talk about things we saw at the 2007 SIOP conference in New York. Here's a smidge:

Later in the conference a panel of other experts on employment testing gathered and discussed how certain scientific and methodological advances in test validation were faring in the field. Specifically, the symposium, entitled “Validity Generalization in the Workplace,” discussed alternatives to traditional test validation strategies that are widely accepted as useful and acceptable by researchers and other experts in the testing industry, but which are sometimes regarded as inscrutable or untested (pardon the pun) by others. Examples included job component validity, validity transportability, and meta analysis. The panelists, all of whom use these validation tactics in their everyday work, explained that they are most often useful and necessary when traditional approaches like criterion-related validation are impossible due to time constraints or the lack of enough incumbents to achieve adequate statistical power for the required procedures.

The incumbents were also forthcoming with many of the sometimes irritating realities of this kind of research, including the fact that one must be able to replicate job analysis procedures for transportability studies and that there was still a certain amount of legal risk involved in these processes given that the courts have yet to build up a strong history of neither support nor opposition for these approaches despite their widespread acceptance among academics and other science-minded practitioners. The presenters also sheepishly provided a somewhat unsatisfying answer to the question of “how close is close enough” when it comes to comparing the components or requirements of two jobs for purposes of transporting validity: “That’s up to you to decide.” It seems there is still a place for professional judgment in the brave new world of alternative validation approaches.

Also of interest for those you who get the print version of TIP, they've changed the cover format to include photographs on the covers. A welcome change, and I since photography has become a hobby of mine I'm hereby making it my life-long goal to get a photograph on the cover. That way I'll be both inside and outside the magazine, which will pretty much make me unstoppable.

  Comments (0) |   Permalink 


Link and Run #4 (7/6/07)

On July 6, 2007 by Jamie Madigan

  Comments (0) |   Permalink 


Robotic Proctor Proctors Robotically

On June 22, 2007 by Jamie Madigan

This story has been making the rounds on technology websites lately. It talks about a device that purveyors of online university degrees can use to help make sure that their students aren't cheating during online tests. The device plugs into one's computer, then during the test it locks things down so that students can't consult Google or Wikipedia for their test answers. But what's more, the device is equipped with a microphone and 360-degree camera that will flag movement and sounds for future review by a proctor in absentia. It does not, unfortunately, instantly vaporize the offending student.

The devices are really pretty cheap at $125, which begs the question of whether or not this kind of roboproctor will migrate from the world of educational testing to the one of pre-employment screening. Unproctored, online testing is a hot topic in the I-O world right now, mainly because many companies are trying to figure out how to leverage information technology and the Internet in general to drive down the costs of testing, especially in far flung locations. Which would you rather do, pay mileage or even airfare for someone to come in to your office or overnight one of these roboproctors to her house and have her take the test at home?

Still, people are already raising the issue of privacy with these devices. And to be sure, I'm not sure I'd want a potential employer to have a 360-degree view of the room where I keep my computer, cluttered as it is. And what about dress? Should I wear my jammies while sitting in front of this thing, or do I have to dress up? And you know, I'm not too crazy about the idea of plugging some strange device into my precious computer and having it install all kinds of wonky stuff that is designed by nature to spy on me.

In the end, I think a more likely use for devices like this wouldn't be at home, but in the field. One testing company with which I've worked has already created test-taking rooms that contain various cameras and microphones instead of stools for proctors to perch upon, with careful hunts for cheaters happening electronically, several testing rooms at a time, from elsewhere in the building.

But I think you could go smaller than that, even. I could easily envision remote testing offices having things like this on hand to oversee tests where more fleshy proctors don't want to make the trip. Testing could be done more easily in retail locations, too, where applicants could walk right in, apply, and test all at once.

Now, if we could just strap a laser death ray onto that thing we'd be in business.

  Comments (2) |   Permalink 


Link and Run #3 (6/15/07)

On June 15, 2007 by Jamie Madigan

  Comments (2) |   Permalink 


An Open Letter to the Internet

On June 8, 2007 by Jamie Madigan

Dear members of the Internet,

Look, you need to stop acting surprised when you post embarrassing stuff about yourself online only to have it come around and cost you a job offer. Employers are increasingly doing cursory Google searches on you while they're bored at work, and they're picking up on stuff like that time you got SOOOO hammered or that time you stole a stole that street sign for some reason that sounded good at the time. Possibly because it was the same time as that time you go SOOOO hammered. And when they've got more applicants than openings, employers may use this stuff to flick your application into the trash bin.

I know, I know --it's not fair, and neither is it even actually a productive (or safe) selection tool for the employer. And sometimes employers (or schools) make really dumb calls and read too much into things, like the woman who was denied a teaching certificate because of a MySpace page with a picture of her wearing a pirate hat, drinking from a plastic cup, and a caption that read "Drunken Pirate." Yes, that's dumb on the university but honestly a lot of the things I see on blogs, MySpace pages, YouTube, Flickr, and Facebook is just mind boggling.

Here's what you need to understand: The Internet is a public place. It's not unlike standing on a street corner and saying things to anyone who wanders by, except that it's all recorded, indexed, and searchable by your prospective employers. So don't say or show anything there that you would be mortified to see show up in a job interview. Or, for that matter, a discussion with your spouse, kids, parents, neighbors, friends, teachers, or dog.

If you must have a personal blog, keep it clean if you want to play it safe. Not every post needs to read like a cover letter, but neither should you willfully incriminate yourself or come across as some kind of semi literate nincompoop. Making hiring decisions based on information found on the web isn't fair and it isn't valid, but it happens.

And if all else fails, you can start a website related to your work so that they actually DO find something relevant.

Sinceerely,

--Jamie

  Comments (0) |   Permalink 


Read more in the Archives

all this copyright until the sun explodes, jamie madigan