Navigation
Search
New Comments
Handy Links
Syndication
« "Eleven Tips on Getting More Efficiency Out of Women Employees" | Main | The link between blinking, speed dating, and speed hiring »
December 2005 issue of Personnel Psychology
On November 15, 2005 by Jamie Madigan
And while we're talking about new journal issues, the December 2005 issue of Personnel Psychology is out and up. It's really the Cadillac of I/O Psychology journals. I don't subscribe to it myself (this website doesn't generate that kind of cash, or any kind of cash really), but the abstracts are available online where you can buy electronic versions of the entire article. Let's take a look at the selection-related one.
An Examination of Impression Management Use and Effectiveness Across Assessment Center Exercises: The Role of Competency Demands
We report 2 studies that examine how promotional candidates use verbal and nonverbal impression management (IM) tactics across several structured assessment center exercises that differ in the competency demands they place on candidates. Based on the competency-demand hypothesis (Shoda, Mischel, & Wright, 1993a, 1993b), it was predicted that IM use would occur most frequently and have the strongest effects on assessor evaluations in exercises that place greater demands on candidates' interpersonal skills than in exercises that depend primarily on technical skills. In both studies, IM tactics were generally used more frequently and there was more variability in IM use for those exercises requiring candidates to display interpersonal competencies (i.e., the role-plays and mock presentation) relative to the exercise that did not (i.e., the tactical exercise). The relationship between IM use and assessor evaluations was also influenced by the competencies assessed by the exercises, and IM use related to both interpersonal and noninterpersonal ratings of performance.
Okay, so people use impression management when they have people to impress. Ever tried to use impression management on a paper and pencil test? Not easy.
Retest Effects in Operational Selection Settings: Development And Test of a Framework
This study proposes a framework for examining the effects of retaking tests in operational selection settings. A central feature of this framework is the distinction between within-person and between-person retest effects. This framework is used to develop hypotheses about retest effects for exemplars of 3 types of tests (knowledge tests, cognitive ability tests, and situational judgment tests) and to test these hypotheses in a high stakes selection setting (admission to medical studies in Belgium). Analyses of within-person retest effects showed that mean scores of repeat test takers were one-third of a standard deviation higher for the knowledge test and situational judgment test and one-half of a standard deviation higher for the cognitive ability test. The validity coefficients for the knowledge test differed significantly depending on whether examinees' test scores on the first versus second administration were used, with the latter being more valid. Analyses of between-person retest effects on the prediction of academic performance showed that the same test score led to higher levels of performance for those passing on the first attempt than for those passing on the second attempt. The implications of these results are discussed in light of extant retesting practice.
Interesting that the restest scores were more valid than the first-timer scores. More variance? At any rate, it suggests that worrying about "practice effects" from retesting isn't as big a deal as we may think. I wonder if the same thing happens with alternate forms that are of equal difficulty but have different items?
A Meta-Analysis of Work Sample Test Validity: Updating and Integrating Some Classic Literature
Work sample tests have been used in applied psychology for decades as important predictors of job performance, and they have been suggested to be among the most valid predictors of job performance. As we examined classic work sample literature, we found the narrative review by Asher and Sciarrino (1974) to be plagued by many methodological problems. Further, it is possible that data used in this study may have influenced the results (e.g., r= .54) reported by Hunter and Hunter in their seminal work in 1984. After integrating all of the relevant data, we found an observed mean correlation between work sample tests and measures of job performance of .26. This value increased to .33 when measures of job performance (e.g., supervisory ratings) were corrected for attenuation. Our results suggest that the level of the validity for work sample tests may not be as large as previously thought (i.e., approximately one third less than previously thought). Further, our work also summarizes the relationship of work sample exams to measures of general cognitive ability. We found that work sample tests were associated with an observed correlation of .32 with tests of general cognitive ability.
So work sample tests are valid after all. Hooray! Good for a citation in my next validity study of a work sample test, I guess. Before I ...you know, validate a specific test, which is what's really most important in that situation.
Existing comments:
Posted by Bryan at December 12, 2005 11:28 AM:
FYI:
1) I contacted Dr. Roth and asked him if they looked at the type of work sample test and/or the type of criterion -- no to both questions. He speculated that most criteria were task-based and raised an interesting question about the value of contextual criteria given the general task-based focus in our field (e.g., Uniform Guidelines).
2) There is another interesting article in this issue (p.1041) that supports using trained job analysts as task-KSA linkers.
Post your own comment:
