Contents
- Theoretical Backdrop
- The Six Basic Personality Profiles
- Real and Situational Sides of The McQuaig Word Survey®
- Administering. Scoring and Interpreting The McQuaig Word Survey®
- Test Fairness
- Research on The McQuaig Job Survey®
- References
- Footnotes
- Bibliographical Notes on Rick D. Hackett
- Summary Tables
- Norms
Download The Full Technical Manual
Prepared by: Hackett & Associates Human Resources Consultants Inc. for the McQuaig Institute® www.McQuaig.com
The McQuaig Word Survey®
The McQuaig Word Survey® (WS) was published in 1967 by The McQuaig Institute® of Executive Development. It was developed under the guiding premise that individuals have characteristic levels on various personality dimensions that are fairly constant (or at least stable in rank order) from one situation to another. Moreover, work behaviour is viewed as largely a function of the complementary match or congruence between the individual’s personality style and the psychological work environment. The WS was accordingly developed as a tool to assist vocational counsellors and personnel managers in matching individual personality types¹ to occupational groups.
Research has suggested that complementary person-work environment links are reinforcing and satisfying and contribute to persistence in those environments (cf. Holland, 1973). On the other hand, incongruence between personality type and work environment is non-reinforcing, increasing the likelihood of suboptimal performance and eventual turnover. The McQuaig Job Survey® (JS) is a companion instrument to the WS and provides a means for describing the WS personality profile most suitable to specific jobs. This instrument and supporting research are described elsewhere in this manual.
Developments in personality psychology since the first release of the WS over 30 years ago affirms that people can be characterised in terms of their enduring dispositional qualities and that this information can be used to improve Human Resource Management and overall organisational effectiveness (cf. Barrick & Mount, 1991; Hogan, 1991; Tett, Jackson, & Rothstein, 1991).
Personality types are syndromes or trait conglomerates. Two persons in the same type category will only share roughly the same traits; rarely will they share precisely the same traits. Type theories are the oldest means we have for classifying the personalities of other people. Furthermore, there are interesting conceptual and empirical convergences in the type theories that have evolved over the past fifty years (Hogan, 1991).
Allport and Odbert (1933) began an important taxonomic effort by compiling a trait lexicon, a listing of English trait descriptive terms.
Their reference list contained over 4500 terms that English-speaking people use to describe each other. Cattell (1947) put Allport’s trait list together with factor analysis in an initial effort to identify the structure of personality. Cattell’s analyses revealed 16 factors (item clusters) underlying the correlation matrix. Fiske (1949) advanced this line of analysis by showing that the structure of personality ratings could be parsimoniously expressed in terms of five broad factors², namely Neuroticism (nervous, self-doubting, and moody versus stable, confident, and effective), Extraversion/Sociability (gregarious, energetic and self-dramatising versus shy, unassertive, and withdrawn),
Conscientiousness (planful, neat, and dependable versus impulsive, careless and irresponsible), Agreeableness (warm, tactful and considerate versus independent, cold and rude) and Culture (imaginative, curious, and original versus dull, unimaginative and literal-minded).
The development of the WS began with generating a large pool of trait related descriptors reflecting four personality domains that capture core aspects of the “big five” factors listed above. These four domains were defined as Compliance (C), Sociability (S), Dominance (D) and Relaxation (R).
The “C” scale draws on specific aspects of the “Conscientiousness” factor (i.e. systematic, methodological, and conscientious) and “Agreeableness” factor (i.e. cooperative). “S” is more narrowly defined than the Sociability/Extraversion factor, with a particular focus on gregariousness. “R” taps traits of the Conscientiousness factor not captured by “C” (i.e. steady, consistent, good with routine, patient). “D” measures the energetic, confidence, assertiveness, effectiveness aspects of the Sociability and Neuroticism factors³.
Self-appraisal ratings were then collected from several hundred successful and unsuccessful executives, supervisors and sales personnel. The profile of successful managers, supervisors and sales personnel across the “D”, “S”, “C” and “R” scales was found to be distinctly different from that of their less successful counterparts.
Included among the defining traits of the “D” scale of the WS are: “very ambitious”, “makes things happen”, “pushes for results”, “opportunist”, and “will to win”. High scores on this scale reflect a competitive, goal oriented, ambitious and self-assured person. It is analogous to Cattell’s “Dominance” factor of the 16PF Questionnaire (Cattell, Eber, & Tatsuoka 1970) and may be considered a desirable attribute for vocations requiring particularly enterprising people (i.e. sales, product manager).
The “S” scale of the WS is defined by such trait descriptors as “entertaining”, “lively”, “friendly”, “sociable”, and “expressive”. It is analogous to Cattell’s “Sociability” factor of the 16PF Questionnaire (Cattell et al., 1970). High scores on this scale reflect individuals most oriented towards working with people, a desirable attribute for vocations requiring considerable interaction with others (i.e. sales, human resource managers, ministers, social workers, and public relations officers).
Trait descriptors of the “R” scale of the WS include “steady”, “patient”, “consistent”, “relaxed”, “reliable”, “easy going” and “good with routine”. This scale can be likened to Cattell’s “Self-Sufficiency” factor (positively) and his “Impulsivity” and “Rebelliousness” factors (negatively) (Cattell et al., 1970). High scores on the Relaxation scale reflect individuals who are planful, and find comfort in routine. Such attributes could be considered particularly desirable for vocations in which there is much routine and predictability (i.e. assembly worker, telephone operator, parking attendant).
Defining the “C” scale of the WS are such descriptors as “systematic”, “methodological”, “conscientious”, “cooperative”, and “good with detail”. It parallels Cattell’s “Conscientiousness” factor (cf. Cattell et al., 1970). High scores on the “C” scale reflect individuals who are very conscientious, cooperative and good with detail — attributes desirable of someone doing telephone surveys, a tour agent, or a registration clerk.
Again, the four WS scales are clearly subsumed under the broader “big five” personality types referenced above. They are more narrowly defined and thereby provide for a more precise matching of personality to the psychological demands of the workplace. Indeed, there is a general recognition among proponents of the five-factor model that personality description can be organised hierarchically and that the five factors are located at the highest level that is still descriptive of behaviour (Cantor, & Mischel, 1979; Goldberg, 1993; Hampson, John, & Goldberg, 1986; John, Hampson, & Goldberg, 1991). The five-factor model simply provides a scientifically compelling framework in which to organise the myriad individual differences that characterise humankind (Goldberg, 1993).
In the development of the WS, 21 trait descriptors of each of the four personality clusters were retained. The decision on whether to retain any particular trait descriptor was based on its conceptual clarity (as reported by respondents) and relatedness to the other descriptors of its own scale. The WS consists, then, of 21 clusters of four descriptors each (one from each personality factor) on which respondents are asked to rank order the trait-descriptors according to the degree to which they are self-descriptive (A = most descriptive; B = 2nd most descriptive; C = less descriptive; D = least descriptive). The scoring consists of giving 5 points for each “A” answer and 3 points for each “B” answer and summing points for each scale (maximum obtainable score on any one scale is 105 while the minimum obtainable score is zero).
Individuals can most precisely be described with the WS by their scores on two or more of these four “basic” scales. In particular, the WS provides for the identification of six primary profile patterns (Table 1). Recommendations for selection, placement and vocational counselling are done on the basis of the individual’s profile pattern and strength, rather than on their standing on any one scale4. Download The Full Technical Manualis an instrument that can be used to assess the psychological demands of the job and facilitate a job-person matching.
Table 15
| HIGH B | LOW B | |
| Generalist | D,S | R,C |
| Pioneer | D | S,R,C |
| Specialist | R,C | D,S |
| Cooperator | S,R,C | D |
| Administrator | D,R | S,C |
| Enthusiast | S,C | D,R |
These six standard profiles do not exhaust all possible profiles, (see “Non Standard Profiles” Section of The McQuaig Survey© Interpreter’s Training Manual). Operational definitions of “high” and “low” scores in determination of profiles are also provided in the McQuaig Survey© Interpreter’s Training Manual.
The Six Basic Personality Profiles
Generalist. A typical generalist is competitive, ambitious and goal oriented.
Welcomes responsibility and authority (high D). Restless and energetic, with a lot of drive and sense of urgency to get things done. Likes variety and works well under pressure (low R). Independent, persistent and decisive, wants to take charge and show initiative (low C). Will be a good motivator and delegator (low C, high S, high D). Sociable and outgoing, a good communicator and persuasive. Understands people and enjoys selling ideas to them (high S).
On the other hand, will need plenty of challenge and opportunity to satisfy him/her and is likely to be dissatisfied unless achieving goals (high D). Dislikes routine and detail although may be good at it (low R, low C). Dislikes close supervision (low C, high D). More oriented towards working with people than with ideas and methods (high S). A temperament typical of many top salespeople, supervisors and managers.
Pioneer. Typically competitive, ambitious and goal oriented, to the point of being aggressive (high D). Restless and energetic, has the drive and sense of urgency to get things done (low R). Independent, persistent and decisive, wants to take charge and show initiative (low C). Welcomes responsibility and authority (high D, low C). Tends to be logical, analytical, practical and realistic, will base decisions on facts rather than feelings (low S).
On the other hand, in his/her need to achieve and with an orientation more towards ideas and methods than people, this person is likely to “step on toes” and hurt sensitive people (high D, low S). Needs constant challenge and opportunity on the job (D over 80). Dislikes routine and detail although may be good at it (low R, low C) and will resent close supervision (high D, low C). This profile is typical of many top salespeople, supervisors and managers, although s/he is likely to be abrasive and have people problems occasionally.
Administrator. A typical Administrator is competitive and goal-oriented, and welcomes responsibility and authority (high D). Independent, strong minded and persistent. Wants to take charge and show initiative (low C). Tends to be logical, analytical and practical, making decisions on facts, rather than emotions (low S). Basically relaxed, patient, steady, consistent and reliable. Can adjust to routine work (high R).
On the other hand, wants challenge and opportunity for advancement (high D). More oriented towards ideas and methods than people, may hurt sensitive people’s feelings at times (high D, low S). Will dislike being closely supervised (low C). Tends to dislike pressure and deadlines (high R). Although this individual does not have the strong drive and social skills typical of many top salespeople, supervisors and mangers, s/he could be a good producer in these areas because of his/her competitiveness and independence.
Specialist. A typical Specialist is very conscientious and cooperative, and follows rules, procedures and policies carefully. Very thorough with detail (high C). Cautious, deliberate, peaceful, a good team player, avoids trouble with people (low D). Relaxed, patient, steady and reliable, enjoys routine (high R). Logical, analytical and work oriented, making decisions on facts, rather than emotions (low S).
On the other hand, not competitive or independent, will not want responsibility for unusual or difficult decisions outside own area of expertise (high C) or for supervising others (low D). Relaxed and somewhat easygoing, will dislike pressure and deadlines (high R). More oriented towards ideas and methods than people (low S). A specialist, would be best in a support role as an expert where steadiness, logical thinking and attention to detail are required.
Cooperator. Typically thoughtful, considerate and peaceful. Works well under supervision and as part of a team (low D). Sociable and outgoing, understands people and likes helping them (high S). Conscientious and cooperative, follows company rules and directions well (high C). Somewhat relaxed, patient and easygoing, will be good with routine (high R) and detail (high C).
On the other hand not competitive or independent, will not want responsibility for supervising others (low D) or for unusual or difficult decisions outside own area of expertise (high C). More oriented toward working with people than with ideas and methods (high S). Lacks the drive and sense of urgency to get things done quickly (high R). This person will be well liked and has a temperament typical of people who do well in support roles in specialised work such as contact, service, public relations, which involve working closely with others.
Enthusiast. Typically sociable, outgoing, a good communicator who likes helping people (high S). Restless and energetic, works well under pressure and can adjust to change and push oneself for results (low R). Tends to be conscientious and cooperative, will follow company rules and directions well (high C). Somewhat thoughtful, considerate and diplomatic (low D, high S), works well under supervision and as part of a team (low D).
On the other hand, more oriented towards working with people than in ideas and methods (high S). Needs variety and dislikes routine, although may be good at it, (low R). Not strongly competitive (low D) or independent (high C), does not want responsibility for supervising others (low D) or for unusual or difficult decisions outside own area of expertise (high C). Overall, a good contact, service, public relations person, this individual will be liked and has a temperament typical of people who are good in support roles, in jobs where they work closely with people.
Transition
A more “transient” and less frequently occurring profile, as compared to the basic six, is that of “Transition”.
Transition. A typical Transition is characterised by nearly equal Dominance and Compliance scores, suggesting the presence of personal problems. A person yielding this profile can be cautious, deliberate, working well as part of a team (low D) and independent, persistent, wanting to show initiative (low C) OR competitive and goal oriented, wanting responsibility for people (high D) and conscientious and cooperative, wanting to follow company rules and directions (high C).
On the other hand, has a problem (equally accepting and independent; D & C equal and opposite). Will be indecisive and inconsistent and will not be effective in handling big responsibilities or big decision-making, often not knowing whether to give in or fight. The Sociability and Relaxation factors, while relevant in the description of an actual individual’s behaviour, are irrelevant in describing a Transition as they can vary among Transitions.
In Summary
The above seven profile types have been shown to be representative of 88% of the individuals responding to the WS®. They provide a classification system for any other less frequently occurring individual profiles. Interpretation of these less frequently occurring profiles are dependent upon the degree and pattern correspondence between that profile and any of these seven profiles. Where it is difficult to match a profile with one of the seven types, the profile can still be described within the WS system. Knowing, for instance, that a profile has similarities to two basic profile types provides useful data for interpretation.
Real and Situational Sides of The McQuaig Word Survey®
As described previously, the WS consists of 21 groups of four descriptive terms, making a total of 84 terms. Each of the four terms in a set (tetrad) was so chosen as to represent one of the four personality traits measured by the inventory, namely Dominance, Sociability, Relaxation and Compliance.
On the front side of the Survey, the respondent is asked to rate each of the four terms in each tetrad either A, B, C, or D, in accordance with how other people think of him/her. After the term which comes closest to describing how other people think of him/her, the respondent is to mark an “A”; after the term which is the second best description of how others think of him/her, the respondent is to mark a “B”, After the third best descriptive term, s/he is to mark a “C”, and after the term which least describes how other people think of him/her, s(he) is to mark a “D”. The score achieved on this side of the Survey is referred to as the “Situational” score.
Having completed the front side, the respondent completes the reverse side of the Survey which consists of the same 21 tetrads and response format (although in different order), but this time the individual is asked to respond according to how each term describes what s(he) is really like. The score achieved on this side of the Survey is referred to as the “Real” score.
Often, the scores respondents receive on the separate factors (D, S, C, R) and their resulting profiles are highly similar between the “Real” and “Situational” sides of the Survey (Miller, Ginsberg & Brien, 1980, Feb.). Where discrepancies between the “Real” and “Situational” sides of the Survey are noted, this suggests that the individual may be attempting to change from the profile reflected on the “Real Side” to a profile more similar to the one reflected on the “Situational” side. These discrepancies are referred to as “Situational Adjustments” Guidance on the interpretation of such discrepancies is provided in The McQuaig Word Survey® Interpreter’s Manual (Section: “The Situational”).
Situational Adjustments: Stretching and Holding Back
Two noted Situational Adjustments are referred to as “Stretching” and “Holding Back”. Individuals who are Stretching display a D score at least 10 points higher on the Situational Side and R and/or C scores at least 10 points lower on the Situational Side than they are on the Real side. An individual who is Stretching is behaving more competitively, more independently and/or with more drive than is normal for him/her.
Individuals who are Holding Back display a D score at least 10 points lower on the Situational Side and R and/or C scores at least 10 points higher on the Situational Side than they are on the Real side. An individual who is Holding Back is behaving less competitively, less independently and/or with less drive than is normal for him/her.
Administering, Scoring and Interpreting The McQuaig Word Survey®
Detailed information on the administration, scoring and interpretation of WS is provided in the McQuaig Word Survey Interpreter’s Manual©.
While not timed, the WS takes about 15-20 minutes to administer. Training in administering, scoring, and interpreting the WS is provided by The McQuaig Institute® of Executive Development Ltd. and its affiliates through the offering of a one-day orientation program where various cases are studied and problems discussed to ensure that a working mastery of the workshop material is obtained. The training is then carried out on a continuous basis by one-to-one coaching over the phone or in person on specific interpretations, by mail and by having the interpreters return for periodic refresher courses.
The McQuaig Institute® also makes available computer software that allows for quicker scoring of WS answer sheets, and provides an interpretation service.
Research on The McQuaig Word Survey®
Research accumulated over the years, while resulting in some refinements to the WS, have supported its conceptual foundation and its usefulness in identifying individuals whose personality types are most congruent with the psychological demands of the workplace.
Word Survey Scale Norms
In a 2012 study of 352,613 individuals employed in different positions in various companies globally, the following Word Survey Scale mean scores were determined:
| “Real” Side | “Situational” Side | |
| D | 47 | 44 |
| S | 47 | 48 |
| R | 33 | 34 |
| C | 43 | 41 |
Word Survey Profile Type Distribution
The 352,613 individuals in the 2012 study were distributed across the following profile types, in order of the most common:
| Generalist: | 20% |
| Enthusiast: | 16% |
| Pioneer: | 14% |
| Specialist: | 14% |
| Persuader*: | 11% |
| Transition: | 9% |
| Cooperator: | 6% |
| Possible Transition**: | 6% |
| Administrator: | 4% |
*The Persuader is not one of the six standard profiles types. The Persuader is a version of the Generalist, defined as having a high D, high S, low R and low C. The differentiating factor is that the Persuader has an S score that is five (5) or more points higher than the D.
** The Possible Transition is defined in a way similar to the Transition Profile, with the D and C scores being nearly equal. The differentiating factor is that the Possible Transition Profile has scores that are not as nearly equal as The Transition Profile. The Balanced Profile type did not impact the overall percentages due to the fact that the total number was significantly less than one percent of the entire sample.
Word Survey Profile Type Distribution (by Situational Adjustments)
In the 2012 sample of 352,613 individuals the results were analysed with respect to whether the individuals were making one of two Situational Adjustments, Stretching or Holding back. Of the entire sample, 24% were holding back and 7% were stretching. The sample was divided into three groups, The Generalist Group (Generalists, Pioneers, Administrators, Persuaders), The Specialist Group (Specialists, Cooperators, Enthusiasts) and the Transition Group (Transitions/Possible Transitions/Balanced).
It was found that 28% of the Generalist Group was holding back while 4% were stretching; 15% of the Specialist Group was holding back while 10% were stretching; 33% of the Transition Group was holding back while 8% were stretching.
Back To Top
Reliability
Reliability refers to the degree to which test scores are free from errors of measurement. There are different ways to assess the reliability of any measurement tool. First, an instrument comprised of items that purportedly measure the same underlying construct or trait should demonstrate high internal consistency. One way to look at this is to say that the score any individual achieves on a randomly split half of the test should correlate highly with the score s/he obtains on the other half of the test. This idea of consistency is extremely important if we wish to infer that individuals can be differentiated on the bases of stable dispositional traits.
Once it is established that responses given to the same test administered at a single time yield high internal consistency, then the next question is whether the trait being measured is stable over time. To assess this requires that the same test be given to the same group of people on two different occasions, with a time period separating administrations. That is, no meaningful differentiation among individuals can be made on personality trait scores that differ widely for an individual from one administration to the next.
Accordingly, any test purporting to measure stable individual dispositions must, in addition to having high internal consistency reliability, demonstrate high test-retest reliability. Quite simply, the person taking the test should be consistent with his/her responses within a single administration and over time.
For the WS, internal consistency and test-retest reliability was assessed at the level of individual factor scores (D, S, R, C) and profile types.
Factor scores. Split-half reliabilities at the level of the four factors were calculated in a study of 418 individuals (predominantly managers) sampled from 14 different companies across a variety of industries (Miller/Ginsburg & Brien, June, 1980). As there are 84 words on each side of the Survey, individual scores for each of the four factors were computed on the first 42 words and correlated with the corresponding factor scores as computed on the remaining 42 words, for both the “Real” and “Situational” sides of the instrument.
These correlations were then corrected by the Spearman-Brown formula (Anastasi, 1988, p.121) for length of test. Correlations for the “Real” side were .87 .76 .78 and .76 respectively for Dominance, Sociability, Relaxation, and Compliance. The respective correlations for the “Situational” Side were .86, 73, 80, and .76 for these same four factors.
Test-retest reliabilities at the level of the four factors were assessed using 231 respondents who were administered the WS twice, where several months separated each administration (Miller, Ginsburg & Brien, Feb. 1980). Test-retest correlations for the four factors were as follows:
| “Real” Side | “Situational” Side | |
| D | .74 | .75 |
| S | .57 | .76 |
| R | .72 | .66 |
| C | .66 | .68 |
Parker, Allen & Co. (Aug. 1983) administered the WS to a sample of 121 adults drawn from five corporations representing food processing, insurance and financial services, manufacturing, retail, and newspaper industries. All 121 were in positions titled “administrative” or higher, and many held supervisory or managerial positions. The WS was again administered to the same group a second time, several months following the first administration. Test-retest correlations were calculated for the four factor scores for both the Real and Situational sides of the WS and are reported below.
| “Real” Side | “Situational” Side | |
| D | .81 | .81 |
| S | .71 | .84 |
| R | .78 | .77 |
| C | .82 | .80 |
Profile types. The reliability of the six basic profile types was also investigated by Parker, Allen & Co. (Aug. 1983). This involved assessing the stability of the profile type for the 121 respondents across two separate administrations of the WS.
The percentage of profile types which were reclassified as the same profile type at retest are shown below.
| Profile Type | Same at Time 2 | # of Respondents |
| Generalist | 70% | 43 |
| Pioneer | 71% | 38 |
| Specialist | 75% | 20 |
| Enthusiast | 67% | 6 |
| Cooperator | 75% | 4 |
| Administrator | 30% | 10 |
| 121 |
An index of the test-retest classification consistency is the Cramer’s V statistic (for nominal data), which is a special case of the Pearson product moment correlation and can be interpreted similarly (Levin, 1977). Cramer’s V was .65 (p < .01).
Because of the small cell sizes resulting from the 121 participants being spread out over the six different profile types, the profiles were reclassified into two “profile families” (A and B) based on their similarities. Family A consisted of “Generalists”, “Pioneers” and “Administrators” and Family B consisted of “Enthusiasts”, “Specialists” and “Cooperators”. The test-retest classification consistency for Family A (N=91) was 89%, and for Family B (N=30) it was 83%. Cramer’s V was .82 (p <.01).
In another study (Evaluation Research Ltd., Jan. 1991), Forms A and B of the WS were administered to 208 persons drawn from a wide variety of job applicants from firms located throughout Canada. (Forms A and B differ only in terms of the arrangement of the items)6. Two profiles were then generated from each side of each Form of the WS, one from the odd numbered blocks, and one from the even numbered blocks (recall that the WS items are presented in blocks of four words each)7. These profiles were grouped into families A (Generalists, Pioneers and Administrators) B (Enthusiasts, Specialists, and Cooperators), and C (Transitions). An analysis was performed to see whether there was a relationship between the two sources of profiles (Odd vs. Even blocks) within each Form of the WS. The results follow:
% of Profiles which Matched According to Family
| Matches | % | |
| 1. Form A (Situational) | Matches 76/106 | % 72%** |
| 2. Form A (Real) | 77/106 | 73%** |
| 3. Form B (Situational) | 68/102 | 67%** |
| 4. Form B (Real) | 74/102 | 73%** |
| (**p <.01) |
Validity
A primary question in personality assessment is: “To what extent does the assessment device measure that which it is intended to measure?” For example, evidence suggests that we can infer from the WS the degree to which an individual is “Dominant”, “Sociable” “Relaxed” and “Compliant” and that the personality types identified by the WS accurately describe an individual’s behavioural tendencies. Another way of saying this is to say that evidence supports the “construct validity” of the WS (see Guion, 1965).
Construct validity. In a study designed to assess the construct validity of the WS, factor scores (D, R, S, C) were obtained from 63 McGill University MBA students and intercorrelated with their scores on the Sixteen Personality Factor Test (Cattell, Eber, & Tatsuoka, 1970). For the Real Side of the WS, the strongest (all p < .05) correlations were as follows:
| Dominance | Sociability | ||
| Dominance | .45 | Impulsivity | .59 |
| Rebelliousness | .34 | Risk Taking | .58 |
| Sociability | .42 | Self Sufficiency | -.27 |
| Relaxation | Compliance | ||
| Risk Taking | -.61 | Risk Taking | -.52 |
| Impulsivity | -.48 | Impulsivity | -.50 |
| Sociability | -.48 | Dominance | -.34 |
| Self Sufficiency | .27 | Rebelliousness | -.28 |
| Intelligence | .27 | Insecurity | .32 |
A very similar pattern of results emerged with regard to the “Situational” side of the WS. The size and direction of these correlations support the interpretations that have been given to the four McQuaig WS factors.
In another study (Miller, Ginsburg & Brien, April, 1981), the construct validity of the WS was assessed when the WS scores of 78 McGill University MBA students were correlated with scale scores on the California Psychological Inventory (CPI; see McAllister, 1986) and the Rathus Assertiveness Scale (RAS). The strongest (r > .30) correlations of the WS (Real) factor scores with the CPI and RAS appear below.
| D | S | R | C | |
| Dominance | .40 | .37 | -.50 | -.41 |
| Capacity for Status | .40 | -.34 | ||
| Sociability | .33 | 49 | -.45 | -.50 |
| Social Presence | .37 | .49 | -.40 | -.48 |
| Self-Acceptance | .41 | -.43 | -.34 | |
| Psychological Mindedness | .30 | |||
| Responsibility | .30 | |||
| Self-Control | -.33 | |||
| Rathus Assertive | .52 | .42 | -.56 | -.54 |
D=Dominance; S=Sociability; R=Relaxation; C=Compliance
The results for the Situational side of the WS were very similar to those reported above and are therefore not reproduced here. While not shown, the highest positive correlation for WS Relaxation was with CPI Self Control (.22), and the highest positive correlation of WS Compliance was with CPI Responsibility (.30) and CPI Femineity (.27).Overall, these results provide strong support for the interpretations that have been given to the four factors of the McQuaig WS.
In another study (Hackett & Associates Human Resources Consultants Inc.; May, 1996), the relationship of the WS to the Myers-Briggs Type Indicator (MBTI) and the NEO Five-Factor Inventory (NEO-FFI; Costa & McCrae, 1992) was assessed on a sample of 206 undergraduate students. Results showed that the WS maps strongly onto two of the “big five” personality factors of the NEO-FFI. Consistent with expectations, NEO-FFI Extraversion correlated -.43 with WS-Relaxation and +.58 with WS-Sociability. Moreover, NEO-FFI Agreeableness correlated +.34 with WS-Compliance and – .40 with WS– Dominance.
In addition, the WS Profile Types could be differentiated based on their overall patterns of scores on the NEO-FFI. Specifically, NEO-FFI scores were significantly different for WS Leaders, Experts and Transitions, as revealed through a 3 (WS-Typing) X 5 (big five personality factors) multivariate analysis of variance (Manova; F10, 294=5.91,
- <.01). A follow-up univariate analysis of variance found WS–Leaders less “neurotic” and less “agreeable” than WS–Experts. Cooperators were more “neurotic” than
Persuaders; Specialists were more “introverted” than all other profile types except Pioneers; and Cooperators were more “agreeable” than Generalists and Pioneers. These findings are also consistent with expectations.
Finally, comparison of WS Types across the 16 MBTI Types revealed that most Leaders were classified as ISTJ’s (serious, quiet, thorough, logical, realistic, dependable, persevering, and independent minded). Most Experts were classified as ISFJ’s (quiet, friendly, responsible, conscientious, stable, thorough, accurate, non-technical, patient with details, loyal, and considerate of others).
Another means of assessing the construct validity of the WS, beyond assessing the relationship of WS scores with other paper and pencil measures of personality, is to see how they relate to observable behaviours of people completing the Survey.
Specifically, if personality traits are expressed behaviourally, and the WS measuresaspects of personality, then measures from the WS should show a predictable pattern of relationships with observable behaviours.
Miller, Ginsberg & Brien, (1980; March), examined the relationship between factor scores on the WS and supervisory checklists of behaviours observed at work. The supervisory checklist assessed characteristics and behaviours across six areas: verbal communications, ability to work under pressure, level of autonomy, interpersonal skills, work habits, and achievement motivation.
Approximately 200 individuals from fourteen Canadian Corporations participated in the study. These people completed the WS and at the same time their supervisors completed the checklists on them. The constructs under investigation were the four WS factors (D, S, R, C), but because they interact to form identifiable profile patterns, the validation results were organised around the profile types. The basic research question then, was whether these identified profile patterns measured and assessed attributes presented within the supervisory checklists.
The results of the study reveal that the WS profile patterns do in fact “map onto” or reflect individuals’ behaviour at work. The basic profile patterns could be distinguished from one another and described in unique ways based on the behavioural checklists. For example, the behavioural description under work habits for the “Pioneer” was: “shows a willingness to delegate authority and decision-making powers. Handles details rather carefully without becoming too embroiled in them and rarely makes requests for extensions of time frames”. In contrast, the behavioural description under work habits for the Enthusiast was: “Sometimes requires assistance from superiors to meet pre- established deadlines. Does not enjoy finely detailed work”.
Yet another approach to assessing the construct validity of the WS is to determine empirically whether the trait descriptors (items) of the WS cluster together to form the four factors (D, S, R, C) that they are thought to reflect.
Accordingly, research involving a sample of 418 persons was undertaken to determine whether the adjectives of the WS encompass these four factors (Miller, Ginsburg & Brien, 1979, Oct.; 1980, June).
A variable analysis (a form of the BC cluster analysis program) was performed on the 84 words appearing on the “Real” side of the WS. The intent of the cluster analysis is to group items that have high positive correlations with one another and thereby form clusters of similar descriptive words. By definition, each cluster contains items that have high intecorrelations; there will be low correlations between clusters. While similar to factor analysis, a factor analysis does not lend itself to the WS data because of the respondents’ choices and ranking of words is done in groups of four items (see Guilford, 1986).
Definers of a particular cluster are those words that form the most collinear subset of items and are also the most independent of definers of other clusters. Using the definers and the expanded list of words within each cluster, six clusters were identified and labelled Dominance, Sociability, Compliance, Reliability, Relaxed, and Individualist. Various indices produced by the cluster analysis programme (e.g. factor coefficients, communalities of defining variables) supported the reliability and validity of these six clusters, however the sixth (Individualist) contributed the least in terms of the total explained variance of items within the WS.
The Dominance, Sociability, Compliance and Relaxed factors of the WS were clearly reproduced in the cluster analysis. The Reliability cluster depicts someone who is systematic and methodical in his or her endeavours, is genial in manner and liked by others. The Individualist cluster describes a person who is stubborn and strong willed.
Clusters 3 (Compliance) and 6 (Individualist) may be thought of as complimentary (with cluster 6 representing the “negative pole” of compliance), as can clusters 4 (Reliability) and 5 (Relaxed). That is, they represent two parts of the original Compliance and Relaxation factors of the WS. A cluster analysis of the items on the “Situational” side of the WS yielded four clusters that were quite similar in interpretation to the same four D,S,R, and C factors.
Criterion-related validity. Along with efforts to establish the reliability and construct validity of the WS, research has investigated the ability of the WS to predict work performance in a variety of occupations. However, because of the ipsative nature of the WS, in most cases attempts to establish predictive validity of the instrument are based on considering the different profile types (rather than individual factor scores) and their relationships to measures of work performance. Individual factor scores, though, are often important as well and are taken into account.
Ipsative scores are used to indicate intra-individual differences in a number of traits. The ipsative nature of the WS is evident from the forced-choice format that pits each word within the four word sets against one another systematically for preferential judgments. The result is that a high preference for one item in any one set contributes to a score for one factor (e.g. Dominance), while at the same time reducing the total score achievable on another factor (e.g. Compliance). It is accordingly impossible for an individual to score very high on all four factors of the WS (see Guilford, 1986). Because they are not independent factors, predictive validity studies should ideally treat the profile type (pattern across factors) as the predictor, which means that the conventional correlation methods (e.g. multiple regression of performance on WS factor scores) are inappropriate.
The summary review of criterion-related studies that follows is organised by the nature of the sample investigated. In many of these studies successful workers were distinguished from less successful workers and WS profiles and factor scores were compared between the two groups.
Sales
In one study of 184 Sun Life Insurance agents (Miller, Ginsburg & Brien, 1980a; April), the agents were classified by their supervisors into one of four different groups, reflecting their level of performance (ranging from “excellent” to “unsatisfactory”). Additionally, the WS profile type of each agent was assessed. A chi-square analysis of cross-tabulations (profile type x performance classification) indicated that profile type was significantly related to the agents’ performance level. Most notably, 75% of all generalists/pioneers were classified as average or above-average performers. In contrast, only 27% of all transition/holding back profile types were classified at this same level.8 The contingency coefficient (a measure of association for nominal data that is interpreted in a manner similar to a correlation coefficient) was .44 (p < .05) for profile type as related to performance level.
This study, using the same design (cross-tabulations and chi-square analysis), also reported a significant association between profile type and agent turnover. Sixty-nine percent of all transitions/holding back were terminated compared to only 27% of all generalists/pioneers (contingency coefficient = .35, p< .05).
Finally, when the factor scores of the Sun agents were compared (Real side), the Sociability scores of the satisfactory performers (average or better) were slightly higher than the Sociability scores of the unsatisfactory performers.
In another study of insurance agents (Miller, Ginsburg & Brien, 1980; Jan.), the WS scores of 61 individuals whose yearly sales were more than $1,250,000 (high producers) were compared to 36 agents whose annual earnings were less than $700,000 (low producers). The top performers had higher Dominance and lower Sociability and Compliance scores on the “Situational” side of the WS and higher Dominance scores on the “Real” side compared to the low performers.
In a three-year tracking study of the longevity and productivity of 184 Sun Life insurance agents (Parker Allen Company, 1982; Dec.), transitions/holding back were especially likely to turnover, with only 11% of all agents so typed still employed with the company after three years. In contrast, 4 out of 10 generalists/pioneers left the company voluntarily over this same time period, with only 38% of these agents having been dismissed. This compares with a dismissal rate of over 50% for the other WS types that were studied.
In another study of life insurance agents (Parker Allen Company, 1984; Oct.) the WS profiles of 62 newly hired agents who stayed with the company for at least six months were studied. Because of the small cell size across individual profile types, agents were divided into three families of WS profiles: generalists/pioneers (N=33); specialists/cooperators/enthusiasts (N=15) and transitions/holding back (N=14).
Generalist/pioneers were more likely to earn high commissions in their first six months of employment than other profile types. This was also apparent for agents with 12-month commission data. Although there were only 19 agents with 12-month commission data, proportionately more generalists/pioneers earned higher commissions (over $10,000) than other profile types during their first year of employment.
To further examine the 6-month performance data, ten “superstar” agents were identified (earned commissions in excess of $8500 with 138% validation). By profile type, the majority (50%) were generalists/pioneers.
Choynowski (Undated) examined the WS scores of 187 salesmen drawn from two different firms. On the basis of ratings provided by their managers, the agents’ performance was classified into one of two groups (“poor or average” vs. “above average or outstanding”). Those classified into the higher performing group obtained higher Dominance scores as well as lower Relaxation and Compliance scores on the “Real” side of the WS.
Parker Allen Company (1981; Sept.) sampled 249 agents from three Toronto branches of a large investment firm. WS profile types were related to an overall supervisory rating (excellent, average, unsatisfactory) and employment status (survivor, voluntarily resigned, involuntarily resigned). Seventy-three percent of agents typed as either generalist or pioneer had been rated as excellent or average performers, compared with only 57% of agents typed as transition/holding back. While only 27% of agents of the former type were rated as unsatisfactory performers, 43% of the latter group were so rated. (Chi-square was significant at p < .01).
Seventy-nine percent of all the transitions/holding back who left the company had been dismissed involuntarily, compared with 51% of all generalists/pioneers who left the company. Finally, a hierarchical regression analysis was run, regressing performance onto nine predictors (occupational test score, WS profile, age, educational level, marital status, number of dependents, sales experience, region and organisational tenure). WS type was coded at two levels (generalist/pioneer vs. holding back/transition). Overall R- Squared was .21 (F = 6.16, N = 218, p < .05), and WS type contributed significant incremental variance to the prediction (p < .05).
Finally, of the generalist/pioneer typed agents who were still with the company (N=24), 58% had been rated as excellent performers, while only 4% were rated unsatisfactory. Of the holding back/transition typed agents who were still with the company (N=23), 35% had been rated as excellent performers and 26% were rated unsatisfactory.
Hackett & Associates Human Resources Consultants Inc. (1995; August) reported WS results for 134 recently hired RBC Dominion Security agents (114 male; 19 female). As all agents were typed predominantly as generalists/pioneers (e.g. virtually no distribution across types), analysis of performance by type was not possible. However, WS “Real C” scores correlated significantly with total assets (r=.30, p <.05) of RBC agents. To examine this relationship further, the distribution of “Real C” scores was divided into approximately equal thirds, and the mean assets of agents falling within each third of the distribution calculated. Those in the lower third (WS “Real C” scores of 18 or less) realised mean assets of $3,720,331.80; those in the middle third (WS “Real C” scores from 19-30) realised mean assets of $5,561,063.80; and those falling in the top third (WS “Real C” scores greater than 30) realised mean assets of $7,637,156.50. These mean differences were statistically significant (F2,89=3.57, p <.01).
The usefulness of the WS in predicting sales performance of real estate agents has also been investigated (see Parker, Allen & Co., 1982a; Oct.). Sales performance measures on 139 new sales agents were collected for their first six months tenure with the company. For purposes of analysis, the four WS profile types studied were generalists (N=30), pioneers (N=23), transitions/holding back (N=45) and enthusiasts/specialists (N=40). Performance measures used included number of listings, total commission dollars earned, # months until first sale, # units sold, and employment status. Analysis involved chi-square of cross-tabulations (profile type x performance).
Results showed that the generalist and pioneer outperformed the agents who were typed enthusiast/specialist or transition/holding back. This was so for the number of units sold and amount of dollar commissions earned. For example, when sales people were divided into two groups on the basis of dollar commission earnings, 52% of the generalists and pioneers fell into the “high performance group”, compared to only 29% of the enthusiasts/specialists. Similarly, for number of units sold, 52% of generalists and 57% of pioneers were classified “high performers” compared to only 38% of transitions/holding back and 47% of enthusiasts/specialists.
Overall, this study of sales agents found the generalists to be particularly well rounded, reliable in sales dollar commissions and unit performance, rather evenly distributed in age, least likely of any of the four profiles studied to turnover, able to generate fast track records in terms of number of months until first sale, and able to generate listings. The transitions/holding back, on the other hand, performed poorly on the two most important measures of performance, the unit and sales commission indices. They also tended to produce fewer listings than other profile types by the end of six months and were less likely to record a sale within their first month of employment.
In a similar study, the performance of 139 real estate agents was tracked and related to WS profile type (Parker Allen Company, 1984; May). Consistent with the earlier study of real estate agents, generalists and pioneers outperformed all other profile types on the sales dollar commission index. Whereas 39% of generalists and 30% of pioneers were classified as “high performers” on this measure, only 4% of the transitions/holding back were similarly classified. With regard to a “units sold” index, slightly more the one- half of the generalists, pioneers and cooperators/enthusiasts performed “high”, compared to 36% for transitions/holding back.
A total of 28% of all first year agents acquired 25 or more listings during the one year time period under study. Compared to this average, pioneers performed above average, with 39% acquiring 25 or more listings. Transitions/holding back performed below this average, with only 20% having 25 or more listings.
Finally, 82% of all salespeople either resigned or were terminated by the end of their first year of employment with the company. While generalists and pioneers showed average levels of turnover (81% and 83% respectively), the transitions/holding back showed the highest turnover (89%), though the differences were not statistically significant.
Overall, this study showed that generalists and pioneers outperformed cooperators/enthusiasts and transitions/holding back on sales dollar commissions earned during the first year of employment with the company. Pioneers also showed a tendency to achieve higher than average numbers of listings. The transitions/holding back performed consistently below average on all three measures of performance, including sales commissions, units sold and number of listings.
In a study of 150 sales representatives of Bretton’s (retail store), WS profiles were grouped into families to overcome small cell sizes within individual profile types (Eshelman & Townsend, 1986; July). The families were generalists/pioneers/ persuaders9 (46%), specialists/cooperators (14%), enthusiasts (32%) and transitions (8%).
The poorest performing group was the specialists/cooperators. Moreover, the poorest performers tended to have higher than average Compliance scores on the “Situational” side of the WS. On the “Real” side of the WS, Dominance scores of good performers were higher than those of average and poor performers. The Compliance scores of good performers tended to be lower than those of the average or poor performers.
In another study of Bretton’s salespeople (Eshelman & Townsend, 1987; Aug), 345 WS respondents were classified into one of three performance groups (very good, average, and poor). Of the 345, 135 had terminated during the course of the study.
Generalists, enthusiasts, and persuaders comprised two-thirds of the sample, with each other profile type having accounted for less than 10% of the entire sample. Generally, generalists and enthusiasts were most likely to have received “average” or “above average” performance (71%). The lowest performance levels were evidenced among the cooperators, but there were too few of them to allow for definitive conclusions.
In yet another study of Bretton’s sales personnel (Evaluation Research Ltd., 1987; Nov.), 321 WS respondents were assessed in terms of their turnover statistics. One hundred and twenty-nine terminated sometime following their completion of the WS. The various profile types were combined into two groups: Leaders (generalists, persuaders & pioneers; N=194) and Experts (specialists, cooperators, enthusiasts; N=127). The proportion of “Experts” remaining with the company exceeded the proportion of “Leaders”, though the two groups received the same pattern of performance ratings whether they were still employed or had terminated. However, “Experts” who were employed full-time and who had low Relaxation scores tended to receive the highest performance ratings.
The WS profiles of 55 sales representatives of Curtis Industries were related to their personal 1985 sales figures. Results showed the highest average sales were achieved by Enthusiasts (mean sales $164,132), followed by generalists ($112,156) and specialists ($93,560). The holding back types had the lowest average sales performance ($74,746).
When average WS factor scores were calculated for each of three performance groups (high, average, and low), low producers exhibited significantly higher dominance scores than average or high producers. Correspondingly, there was a tendency for low producers to have lower average compliance scores than the average or high producers (for both the “Real” and “Situational” sides of the WS). This finding is inconsistent with the vast amount of previous research showing that high Dominance and low Compliance is often associated with success in sales.
In an effort to assess the validity of the WS in predicting sales performance in countries outside Canada, international studies were undertaken.
In one such study, a sample of 62 Life of Jamaica agents were studied (Parker Allen Company, 1983; Sept.). Performance measures were: number of policies written, premium income in dollars, and dollar commission earnings. Because of low cell frequencies, the agents were classified into three groups based on their WS profiles: generalists / pioneers (N=21); specialists/cooperators and enthusiasts (N= 22); and transition/holding back (N=19). It is common to group profiles in this manner because the profiles within each group closely resemble one another. These groups were formed to provide an adequate number of profiles for group statistical analyses.
The results were very consistent with those reported for the North American samples. Specifically, over 50% of the generalists/pioneers were earning commissions of $21,000 or more annually, compared to only 21% of the transitions/holding back. While only 19% of generalists/pioneers earned less than $13,000 in annual commissions, 53% of the transition/holding back fell in this category. On average, the generalists/pioneers earned $25,073 in annual commissions, compared with average annual commissions of $14,994 for the transition/holding back agents. This difference is statistically significant (p< .05).
The average premium income of the generalist/pioneers ($55,076) was more than $20,000 higher than the average premium income achieved by the transitions/holding back ($32,377). Of the 22 agents with premium incomes of $46,000 or more, 11 were generalists/pioneers, and only four were transitions/holding back.
There was also a clear relationship between number of policies written and profile type. The generalist/pioneer group was more likely than either of the other two groups to write more than 60 policies annually. The transitions/holding back performed most poorly on this index, with 68% of them writing fewer than 60 policies a year.
On all the performance measures reported so far, the specialists/cooperators/ enthusiasts fell in between the generalists/pioneers (high performers) and the transitions/holding back (low performers). Finally, generalists/pioneers had the highest levels of conservation (retaining their clientele). Fifty-two percent conserved 90% or more of their policies.
In another study, 117 British Prudential insurance agents were given the WS and their work performance reviewed (Parker Allen Company, 1985; March). WS profiles were used to predict agent performance. Accordingly, from the WS profiles, three groups of agents were identified: high potential (N=29); medium potential (N=21); and low potential (N=67). Results indicated consistent correspondence between the predicted and actual performance (e.g. a 2/3rds “hit rate”).
Sixty-six percent of agents classified by WS results as “high potential” agents were classified by Prudential as “very good”. Ninety percent of all agents classified as high or medium potential by the WS were independently evaluated as “average or above average” in overall personal performance appraisal according to Prudential classifications. In contrast, only 16% of the agents classified by WS results as “low potential” agents were evaluated as very good by their company. There were also statistically significant relationships (in the expected direction) between WS predictions and other specific subcomponents of performance (e.g. job knowledge, sales product knowledge, sales, ability, motivation). Moreover, agents who were considered high potential from the WS earned higher average salaries than did their low potential counterparts.
Overall, the predictive validity of the WS is underscored in this study by the following. Seventy-five percent of all agents identified by the WS as high potential received Prudential Assurance Company evaluations of “good” or “very good”, compared to one half of all medium potential agents and one-sixth of all low potential agents (only 7% of the agents identified as high potential were given poor overall performance appraisals by their company; 14% of medium potential agents were rated as poor performers, but nearly one-half of the low potential agents were rated as poor performers). The chi-square analyses of the cross tabulations (4 levels of performance x 3 levels of potential) was statistically significant (p < .05, N = 117).
This study sampled 287 British Prudential sales agents (Eshelman & Townsend, June, 1987b). Sixty-seven of these agents were terminated from the company, while 220 remained employed. The breakdown by WS profile type was: generalists (87), persuaders (51), pioneers (29) and enthusiasts (20), with other profile types represented by 14 or fewer agents.
Among the agents that were still employed, the highest performance ratings were associated with three profile types (cooperators, generalists, specialists). More than 60% of these agents were rated as “very good” or “good” performers. The enthusiasts were the poorest performers, with 65% of them rated as average or poor performers, and only one-third (35%) achieving a performance rating of “good” or “very good”. While cooperators and specialists have classically not been associated with high sales in previous studies, their performance here might be explained by the fact that their job with this company required considerable attention to detail and paperwork.
Lewis (1991) conducted a validation study of the WS with a British sample of 244 individuals employed by Barclays Financial Services (BFS). The validation study consisted of all those who joined BFS between January 1, 1990 and November 1, 1990, and their performance was tracked for the six months from March to August 1991.
Following cleaning of the data, sample size was reduced to 219. Performance measures included sales, corrected for regional differences, and percentage of life assurance sales eventually taken up by the client.
Individuals falling within a boundary of scores across the four factors of the WS (D=60-90 points; S=30-70 points; R=15-35 points and C=10-35 points) were predicted to outperform individuals falling outside this profile. Of those identified as falling within these boundaries, 32 were high performers and 29 were low performers. However, of those falling outside, 20 were higher performers and 43 were low performers. Accordingly, those within the profile boundaries were more likely to be high performers than were those outside (chi-square = 4,64, p < .05). This was corroborated by the observation that the mean performance on the criterion measure of individuals falling outside the profile was 9.53, compared to a mean of 10.56 for those falling within the profile boundaries.
This indicated an overall performance benefit of approximately 11% to be gained from using the WS as a selection instrument.
Eshelman & Townsend (1987; March) studied the WS scores and performance of 217 American Kraft territory managers who occupied sales positions. WS profile types included transitions/holding back (N=58), generalists (N=45), persuaders (N=32), pioneers (N=25), specialists (N=20), enthusiasts (N=19), cooperators (N=10) and administrators (N=8). Generalists and pioneers outperformed all other profile types as territory managers. Only 25% of all profile types were rated as high performers by Kraft. Significantly higher than average proportions of generalists (42%) and pioneers (44%) were high performers. From another perspective, only 8% of pioneers and 13% of generalists were rated as low performers.
The performance levels achieved by transitions/holding back, persuaders, and specialists were typically low. Only 21% of transitions/holding back, 19% of persuaders, and 10% of specialists were rated as high performers. Twenty-eight percent of transitions/holding back, 25% of persuaders and 20% of specialists were rated as low performers.
On both sides of the WS, and particularly on the “Situational” side, the Dominance score of “high” performers was significantly greater than that observed for “average” or “low performers”. High performers also displayed significantly lower average Relaxation and Compliance scores as compared to the “average” or “low” performers.
Back To Top
Managers
Research on the WS has also used managers as a validation sample.
Nishisato (undated), for example, investigated the predictive validity of the WS in discriminating between successful and unsuccessful managers. The sample consisted of 102 managers drawn from fifty companies throughout Canada. Each manager was rated by senior personnel within their respective companies as either “successful” or “unsuccessful” performers. A linear discriminant function analysis was done between these two groups, treating the four factor scores of the WS (D, S, R, C) as the discriminating variables. The resulting discriminant function correctly classified the managers into one of the two performance groups (based on their WS factor scores) 92.9% of the time. Additional analyses showed that each of the four factors contributed significantly to the classification (discriminant function).
A two-way (two levels of performance x 4 factor scores) analysis of variance with repeated measures on the scales was also conducted. Results indicated a significant (p < 01) tendency for successful managers to obtain higher scores on Dominance and Sociability, and lower scores on Relaxation and Compliance than unsuccessful managers.
In another study (Eshelman & Townsend, 1988; Aug.), 75 Canadian managers were sampled from a large grocery store chain (Miracle Food Mart and Ultra-Mart). Their performance was rated by their employers as “excellent”, “good”, “average” or “below average” and their WS types were identified. Twenty-three (31%) were rated as excellent, 24 (32%) were rated as good, 19 (25%) were average and 9 (12%) were rated below average.
Twenty-two managers (29%) were classified as pioneers, representing the largest profile group. There were 15 generalists/persuaders. Because these two profiles resemble one another and there were so few persuaders in the entire sample, they were combined for the analyses.
Findings suggested that the pioneers performed best as store managers. Moreover, the individual factor scores that were most associated with successful managers were higher than average Dominance scores and lower than average Relaxation and Compliance scores on the “Situational” side of the WS. On the “Real” side of the WS, higher than average Dominance scores (up to the mid-70’s) and lower than average Compliance scores were associated with stronger performance among the store managers.
In analysing profile types by performance, conclusions were drawn only for those types represented by 15 or more managers (pioneers, generalists/persuaders, holding back). The highest performance ratings were associated with pioneers, with 77% of them having been rated as either “excellent” or “good” performers.
Eshelman & Townsend (1991; Sept.) investigated the relationship of WS scores to the performance of 188 Kmart Assistant store managers. Performance measures were provided by the company, with 38% rated “outstanding/good”, 49% rated “satisfactory”, and 12% “unsatisfactory” (ratings were missing for 1% of the sample). The highest performers were most concentrated among the generalists (59%), followed by pioneers (48%). The number of individuals within the other profile groups was too small to allow for reliable statistical analyses.
Average WS factor scores for the “Real” and “Situational” sides of the Survey followed similar patterns. On average, the assistant store managers rated as outstanding/good/satisfactory had higher Dominance scores and lower Compliance scores than did the assistant managers rated as unsatisfactory performers. The group rated outstanding/good achieved the highest average Dominance score, the satisfactory group achieved the highest average Sociability scores, and the unsatisfactory group achieved the highest Relaxation and Compliance scores.
Eshelman & Townsend (1992; Feb) sampled 127 Kmart store operational managers. Performance ratings of these managers as provided by Kmart were distributed 51% “good”, 36% “satisfactory”, and 13% “unsatisfactory”. Pioneers and generalists were the higher performing managers, with 67% and 63% of each group respectively rated “good” performers. Additionally, operational store managers rated as good or satisfactory had higher Dominance scores and lower Compliance scores than managers rated as unsatisfactory. The group rated “good” achieved the highest average Dominance scores and the unsatisfactory group achieved the higher Relaxation and Compliance scores.
In another study of managers, Eshelman & Townsend (June, 1987a) investigated WS scores and performance of 86 managers of Zellers stores (large retail chain). The performance of the managers was rated as “best”, “average” or “lowest”. The WS types that were represented in sufficient numbers to allow for reliable analyses were pioneers (26%), specialists (15%), generalists (14%) and transitions (15%). The highest performance ratings were associated with pioneers, with 68% of managers of this profile type classified within the “best” performance category. Additionally, those managers rated as best performers achieved higher Dominance and lower Relaxation scores, on average, than did the average or lowest performing groups of managers, on both sides of the WS.
Given the nature of managerial/leadership demands placed on chief executive officers (CEOs) and presidents of corporations, it was hypothesised that individuals holding such positions would have temperament traits similar to those of generalists and pioneers as measured by the WS. A study by Parker Allen Company (1984; March) assessed this. In particular, WS profiles on 75 CEOs/company presidents from a variety of organisations were collected during 1983. The sample was multi-national: 69.3% were from Ontario, 13.3% from the rest of Canada, 13.3% from the U.S., 2.7% from the U.K., and 1.3% from Jamaica. However, it is most appropriate to generalise the study findings to Canadian CEO’s and presidents, given their extensive representation in the sample.
One-half of the sample was affiliated with manufacturing, 18.7% were in service industries, and another 12% were in finance/insurance/real estate.
Consistent with expectations, over 75% of the CEOs/presidents were either pioneers (41.3%) or generalists (37.3%). Another 16% were administrators (accumulatively, greater than 90%).
Research on the usefulness of the WS in predicting managerial performance of non-Canadian samples has also been conducted.
Eshelman & Townsend (1990; Jan.) sampled 39 British Sainsbury store managers, who had been classified by their employer as either “top” (51%) or “bottom” (49%) performers. Sainsbury is a large retailer, employing approximately 70,000 people, with a chain of grocery stores operating throughout the U.K. The store managers most likely to be “top” performers among the WS types analysed were pioneers (67%) and administrators (80%). The profile types most represented among the “bottom” performers were specialists (80%) and holding back (60%).
Among top rated store managers, pioneers were represented most often (30% of all top rated store managers), followed by generalists (20%) and administrators (20%). Specialists are most often included among bottom performing mangers (21% of all bottom performing managers).
Top performing store managers achieved higher average Dominance scores than did their bottom performing counterparts, for both the “Real” and “Situational” side of the WS. Bottom performing managers had higher Sociability and Relaxation scores than top performing managers on the “Situational” side of the Survey and higher Compliance scores on the “Real” side of the Survey.
Hackett and Associates (2005a) reported WS data on 763 business leaders; 152 from Canada and 611 from Australia. This sample consisted of 635 Chief Executive Officers and Presidents (senior executives) and 128 managers. The WS types, profiles and dimension scores of the senior executives and managers were perfectly consistent with expectations. Most business leaders (senior executives and managers) were typed as WS Leaders, whether Canadian or Australian. However, the representation of Leaders among senior executives (77%) exceeded their representation among managers (60%). There were also greater percentages of Experts and Transitions among managers than among senior executives. The senior executives were profiled predominantly as Generalists and Pioneers and least profiled as Transitions and as Specialists. The mean WS dimensions scores of the business leaders (managers and senior executives combined) were highest in Dominance, next highest in Sociability, with substantially lower levels of Relaxation and Compliance.
Non Sales / Non Managerial Samples
Validation studies of the WS have been done on samples other than sales or management positions.
Miller, Ginsburg & Brien (1980b; April) sampled 25 transit workers of a Canadian transit authority, whom were divided into three groups on the basis of their job performance: “above average” (N=6), “average” (N=9) and “below average” (N=10). For purposes of analyses, these three groups were collapsed into two groups: “average/above average” (N=15) and “below average” (N=10). On both sides of the Survey, the Sociability and Relaxation factors were the most important ones in distinguishing between the two performance groups. Higher Relaxation scores and lower Sociability scores distinguished the higher performing group.
In another study (Parker Allen Company, Oct. 1982b), the WS profiles and performance of 180 Dun and Bradstreet business reporters were investigated. The five WS profile types studied included generalists, pioneers, holding back, transitions, and specialists/enthusiasts/cooperators. WS type was related significantly to turnover. While generalists and pioneers were least likely to turnover, holding back and specialists/enthusiasts/cooperators were most likely to turnover. There was no statistically significant association between supervisory ratings of monthly performance and WS profile type.
Parker Allen Company (1983; Jan.) also examined the relationship of WS profiles to the job performance of 54 Amdahl field and service engineers. With service engineer performance described as “superior”, “average” or “below average” by Amdahl personnel, cooperators, specialists and administrators displayed the highest proportions of “superior” performance”. Service engineers with superior performance also achieved lower Dominance and higher Relaxation scores than service engineers with “average” or “below average” performance. The “average” profile of a superior service engineer most closely resembled that of a specialist or administrator, while the “average” profile of an “average/below average” performing service engineer was most similar to that of a generalist.
Among field engineers, highest performance ratings were most achieved by administrators, specialists, and transitions. Field engineers with superior performance also achieved lower Dominance and Sociability scores, but higher Relaxation scores, than did the field engineers with “average/below average” performance ratings. The “average” profile of a superior field engineer most closely resembled that of an administrator or specialist, while the “average” profile of an “average/below average” field engineer was most similar to that of a generalist.
For the combined group of service and field engineers, Dominance correlated negatively, and Relaxation correlated positively with performance. The traditional interpretation of a high Relaxation score is that the person is relaxed, steady, patient, consistent, reliable, good with routine, cooperative and unassuming. People low in Dominance are usually described as cautious, deliberate, working well under supervision, good with detail and routine, avoids trouble with people, modest and unassuming. This profile seems to be most suited to the engineer.
The WS has also been used in a validation study involving 89 work wear routemen (Parker Allen Company, 1985; Feb.). Criterion measures included “overall performance ratings” (above average, average, below average), reasons for termination (involuntary, voluntary, or health reasons), and length of employment. WS results were documented for ten employees who “turned over” prior to six months of employment, 19 employees who “turned over” after at least six months of employment, and 60 employees who were still with the company. Overall, then, 29 terminated employees and 60 currently employed routemen comprised the sample.
As with previous small sample studies, three groups of WS profiles were formed (Group 1: generalists, pioneers, administrators, and persuaders, N= 28; Group 2: enthusiasts, specialists, cooperators, N=40; and Group 3: transitions/holding back, N= 21). Overall, those in group 3 tended to have higher involuntary dismissal rates (75%), and those in group 2 had the highest voluntary resignation rates, as pertains to turnover after at least six months of employment.
Analysis of relationships between WS profile types and performance relied upon categorical evaluations. Among all still employed routemen, 25% were evaluated “above average”, 70% were rated “average” and 5% were considered “below average”. The largest proportions of “above average” performers were from group 1 while the largest proportions of “below average” performers were from groups 2 and 3.
A study was undertaken to assess the usefulness of the WS within the transportation industry (Parker Allen Company, 1981; June). Specifically, the WS data on 86 Canadian railroad reservation clerks were collected. WS profiles were classified into four groups, generalist/pioneer, specialist, enthusiast, and transition. Performance data consisted of supervisory ratings of each employee as either “above average” or “below average”. Past supervisory performance assessment across eight areas was also available (job knowledge, sales effort, initiative, interest in the job, interpersonal skills, punctuality, absenteeism, and problem behaviour). These ratings were summed to produce a “past supervisory assessment score”. Present supervisory ratings across these 8 areas were also available, and when summed produced the “present supervisory assessment score”.
With performance categorised dichotomously (above average, below average), performance level was unrelated to profile type. With performance expressed as past supervisory assessment score, performance differed significantly by profile type. Ranked from highest to least in performance are generalists, specialists, enthusiasts, transitions. Performance did not differ significantly across profiles when the “present supervisory score” was used as the criterion.
Three WS factor scores were significantly associated with performance level, when performance was expressed dichotomously (below average, above average). Relaxation (on both the Real and Situational sides) related positively with performance (r = .26, Real side, p < .01; r = .25, Situational side, p < .05). Sociability on the Real side of the Survey was negatively correlated with above average performance, indicating that lower than average Sociability scores were characteristic of above average performers (r = -.20, p < .05).
Research by Hackett & Associates (November 2000) has shown that the profile of entrepreneurs is one consistent with expectations. In this study, the WS was administered to 1509 members (239 females, 1270 males) of the Young Entrepreneur Organisation (YEO). YEO members operate businesses with revenues of $1,000,000 or more and are 38 years of age or less. A strong majority of these YEO members (78.7%) were shown to have a WS “leadership” profile (Generalist, Pioneer, Persuader and Administrator). The predominance of the leadership profile was evident among males (80.2%) and females (70.7%). Moreover, YEO members had a WS factor profile characterised as highest in Dominance, next highest in Sociability, with substantially lower levels of Relaxation and Compliance scores.
The above profile was replicated with a sample of 1654 entrepreneurs surveyed from the U.S. (1509) and the U.K. (145) (Hackett & Associates, 2005b). Approximately 79 per cent of these entrepreneurs were typed as WS Leaders (Generalists, Pioneers and Persuaders). Moreover, they scored highest in Dominance, next highest in Sociability, with substantially lower levels of Compliance and Relaxation. For males and for females, WS profiles of U.S. and U.K. entrepreneurs were distributed similarly.
Hackett & Associates (2009) sampled 248 non-managerial employees in Underwriting (N=143) and Claims (N=105) in the insurance industry. All employees were rated on Performance (Outstanding, Superior, Good, Below) and Potential (High, Promotional and Lateral). In the underwriter group the Specialist is the most common profile in the Good (35%) and Superior (30%) groups, c2 (7, 129) = 16.5, p < .05. The Superior category has a higher percentage of Leader profiles (47%) than Expert profiles (40%), while the Good category has a higher percentage of Expert profiles (68%) than Leader profiles (24%), c2 (1, 116) = 8.8, p < .01.
In the underwriter group a larger portion of individuals with Lateral Potential have Expert profiles (71%) than Leader profiles (24%), while more individuals with Promotion Potential have Leader profiles (46%) than Expert profiles (36%), c2 (1, 118) = 10.5, p < .001.
In the claims group the Specialist is the most common profile in both the Good (27%) and Superior (31%) categories. (though not significantly more common than the Enthusiast.
The Good category has a higher percentage of Expert profiles (59%) than Leader profiles (33%). The Superior category also has a higher percentage of Expert profiles (54%) than Leader profiles (32%).
In the claims group individuals with Lateral Potential have a greater percentage of Expert profiles (55%) than Leader profiles (38%). Individuals with Promotion Potential have a greater percentage of Expert profiles (65%) than Leader profiles (26%). c2 (1, 23) = 3.52, p < 06.
Back To Top
Test Fairness
A personnel test is deemed unfair, if as used operationally it does not provide an equal opportunity for all test-takers to demonstrate their capacity to perform the job in question. Accordingly, a test should not screen out disproportionate numbers of a group (e.g. females) without evidence that members of that group perform less well on average in the job. Research has been conducted to assess whether performance on the WS differs by gender, age, race or language (French vs. English).
Parker Allen Company (1983; May) compared the WS performance of males (N=40) and females (N=28) who were enrolled in the Masters of Business Administration programme at McGill University (Montreal). One statistically significant difference was found in the mean performance of males and females across the four WS factors for either side (Situational or Real) of the Survey. Specifically, females scored slightly higher than males in terms of Sociability on the “Real” side of the Survey. However, these differences did not affect profile interpretation. When distributions of male and female profiles were compared, non-statistically significant differences emerged. These results suggest that both males and females have equal probabilities of being classified within any of the main profile types (e.g. specialist, generalist, pioneer, transition, holding back). As it is recommended that personnel decisions be based on the entire profile rather than on specific factor scores, this finding suggests that the WS will not adversely affect females.
Differences in WS scores of a larger group of male (N=73) and female (N=131) McGill University students were investigated by Parker Allen Company (1983; July). Once again, a chi-square analysis of profile type by sex failed to reach statistical significance.
The profile types (five groups) represented in this analysis were generalist, pioneer/administrator, transition, enthusiasts/cooperators and specialists. Additionally, no differences were noted between males and females when mean factor scores (D,S,R,C) were compared for the “Situational” and “Real” side of the WS Parker Allen Company (Oct. 1982a) similarly found no differences in the WS profile types of recently hired male (N=64) and female (N=75) real sales agents, nor were there differences associated with age. The four WS profile groups considered in these (chi- square) analyses were (generalists, pioneers, transitions/holding back and enthusiasts/specialists). Age was classified into three groups, under 30 years, 30-39 years, and 40 and over.
In the earlier reported study of Dun and Bradstreet business reporters (Parker Allen Company, Oct. 1982b), possible differences between male (N=126) and female (N=54) WS profiles were again assessed. A chi-square analysis of profile types (five levels) by sex (two levels) failed to reach statistical significance. Profile types examined in this study were generalists, pioneers, transitions, holding back, and enthusiasts/cooperators/specialists.
Eshelman & Townsend (1987; March) tested for sex differences across WS profiles for 217 Kraft territory managers (82% male; 18% female). Again, sex was unrelated to profile type, based on a sex (2 levels) by profile (8 levels) chi-square analysis. The profile types examined in this study were generalists, pioneers, specialists, cooperators, administrators, enthusiast, persuader and transitions/holding back. Similarly, chi-square analyses revealed no statistically significant differences in WS profile types by education (3 levels; high school, post high school, unknown), age (3 levels; 21-30, 31-40, over 40) or race (2 levels; white, n = 202, nonwhite, n=14).
In their study of 1509 (1279 males, 239 females) members of the Young Entrepreneurs Organisation (YEO), Hackett & Associates (November 2000) found a predominance of WS leaders among both male (80.2%) and female (70.7%) YEO members. There were a larger percentage of females as Experts (18.4%) compared to males (11.1%). On the WS factor scores, males scored higher (p. < .05) on Dominance than did females (for both Real and Situational scales) and, consistent with previous findings, females scored higher (p. < .05) on Sociability than did males (for both Real and Situational scales). While males scored higher than females on Relaxation (for both Real and Situational scales), the magnitude of the difference was small (approximately 3), compared to the mean differences between sexes in Dominance and Sociability (7.8-9.4). The statistical significance of a difference of 3.0 may be attributable to the high statistical power associated with a large number of YEO members on which the analysis was performed (1270 versus 239). In such cases, more weight is to be given to the magnitude of the difference, than in the statistical significance when interpreting the results. The larger percentage of females as Experts, compared to males, may be because their stronger Sociability leads to a more participative leadership approach with an emphasis on teamwork.
Eshelman & Townsend (1991; Sept.) investigated possible differences in WS profile by age, sex, race, language (English vs. French), education and marital status. While no statistically significant (p < .05) differences were uncovered, low cell frequencies in many of the Tables prohibit definitive conclusions.
Lewis (1991) conducted a validation study of the WS with a British sample of 244 individuals employed by Barclays Financial Services (BFS). Individuals falling within a boundary of scores across the four factors of the WS (D=60-90 points; S=30-70 points; R=15-35 points and C=10-35 points) were predicted to perform better on a number of work measures than individuals falling outside this profile. Results indeed confirmed that those within the profile boundaries were more likely to be high performers than were those outside (chi-square = 4,64, p < .05). Further analyses were done to test for possible sex differences. Inside the “desirable” profile boundary, there were 94 males and 21 females, as compared to 105 males and 23 females falling outside the boundary. This produced a chi-square value of almost zero, indicating that there was no significant difference between the genders in terms of whether they fell within or outside the profile boundary.
Eshelman & Townsend (1992; March) examined the relationship between major profile types and demographic variables, including gender (1,357 males, 456 females), race (1496 “whites”, 138 “blacks”, 74 “other”) and geographic location (n=1714). The major profile types examined were leaders, experts and transitions. Leaders represented 85% of the total sample (1533); Experts comprised 12% (214), and Transitions represented 4% (66). Leaders included generalists, persuaders, pioneers, and administrators and Experts included enthusiasts, cooperators, and specialists. Respondents in this study were applicants for sales positions with Fridel Alcatel.
The results of this study clearly revealed that representation by race and sex across the major profiles was proportional to their representation in the total sample, and that the Survey could be used with sales position applicants in this setting without potential adverse impact. For example a total of 80% of all Experts were males, compared to 75% of the total sample. A total of 26% of all Leaders were females, compared to 25% of the total sample. Seventy-four percent of all Leaders were males, compared to 75% of the total sample.
Proportions of “white”, “black” and “other” Experts were very similar to proportions observed in the total sample. Among Experts, 85% were white, compared to 88% of the total sample. A total of 11% of Experts were black, compared to 8% of the total sample. Finally, 5% of Experts were classified as “other”, compared to 4% of the total sample.
Proportions of “white”, “black” and “other” Transitions were also very similar to proportions observed in the total sample. Among Transitions, 83% were white, compared to 88% of the total sample. A total of 12% of Transitions were black compared to 8% of the total sample. The remaining 5% of Transitions classified their race as “other”, compared to 4% of the total sample.
In another study (Parker Allen Company, 1983; July), 204 McGill University students were classified by profile type to identify similarities and differences between the French and English versions of the WS. In this sample of undergraduate and MBA students, 99 copies of the French version of the WS, and 105 copies of the English version of the WS, were completed.
When WS’s were classified by profile type (Generalist, Pioneer, Transition, Enthusiast, Cooperator, Specialist), the distribution of profile types identified was similar for the French and English versions (as revealed by a chi-square analysis). In analysing the French and English mean factor scores, a repeated measures ANOVA revealed that the francophones were somewhat more compliant, less relaxed and (on the “Situational” side) more sociable than the anglophones. Although these differences in mean factor scores were statistically significant, they did not influence profile interpretation.
French and English WS’s were regrouped to identify similarities and differences for males and females. When surveys were classified by profile type, a chi-square revealed no statistically significant differences in the distributions of male and female profile types. Additionally, no differences were noted for males and females when mean factor scores (D,S,R,C) were compared for the “Situational” And “Real” side of the WS (based on a repeated measures analysis of variance).
In a study of U.S. University students (see Hackett & Associates Human Resource Consultants Inc; 1994, June), WS scores of 121 Whites, 89 African Americans and 74 Mexican Americans were examined for possible race and sex differences. Results indicated no differences in WS profile types either by sex or by race. Differences were noted only in WS factor scores. Specifically, the scores for males exceeded that of females for Dominance, on both the “Situational” (male = 44.07, female = 38.60, F1,246 = 6.03, p < .05) and “Real” (male = 46.48, female = 40.37, F1,246 = 6.69 p < .05) sides of the Survey. Also, the mean score of females (41.00) exceeded that of males (36,32) for Compliance (“Situational Side”, F1,245 = 5.98, p < .05). There were two significant effects for race. Whites scored higher (mean = 52.25) in Sociability (“Real Side) than did Mexican Americans (mean = 45.28; t176 = 2.60, p < .01), and African Americans scored higher (mean = 47.54) than Whites (mean = 40.04) on Dominance (t186 = -2.54, p < .01). Although these differences in mean factor scores were statistically significant, they did not influence profile interpretation.
As decisions based on WS results rely primarily on the WS profiles rather than factor scores, the results of this study suggest that WS based decisions would not adversely affect Whites, African Americans, or Mexican Americans, nor members of either sex.
Gender differences were again examined. Responses to a Spanish translation of the WS as provided by 224 Hispanics were compared to WS responses of 83 African Americans, 118 American Whites and 74 Hispanics, all of whom completed the English version of the WS (see Hackett & Associates Human Resources Consultants Inc., 1996, May). There were no gender differences in WS responses in any four of the groups, either by WS-Type or WS-Dimension.
Hackett & Associates Human Resources Consultants Inc. (2001; April) compared WS responses of 350 business students across three countries – Canada (n =105), China (n=136) and India (n=109). The English version of the WS was administered in all cases without translation. Results showed that there were no differences in responses between Indian males and Indian females; no differences in responses between Chinese males and Chinese females; no differences in responses between male and females across all three countries; and no cross-country differences in WS profiles or WS types.
Hackett & Associates (2005a) reported WS results for 763 business leaders (635 senior executives; 128 managers). 152 were from Canada and 611 from Australia. Most were typed as WS Leaders, followed Transitions and Experts. This was observed for the senior executives and for the managers; for Canadians and for Australians separately, and for the combined Australian and Canadian group. Canadians were distributed similarly to the Australians across WS types. The distribution of males and females across WS types did not differ for the senior executives, for the managers, of for the combined sample.
For the combined sample, senior executives were profiled predominantly as Generalists and Pioneers. Australians and Canadians were not distributed any differently from each other over the nine WS profiles, nor were the males and females. Mean WS dimension scores of business leaders, whether Australians or Canadian, male or female, or senior executives or managers, showed a pattern highest in Dominance, next highest in Sociability, with substantially lower levels of Relaxation and Compliance. This pattern was repeated for the combined Australian and Canadian group. Canadian (and male Canadian) senior executives scored higher than Australian (and male Australian) senior executives on Compliance-Situational. Female senior executives scored higher than males on Sociability (Situational and Real) and lower on Relaxation (Situational).
Canadian managers scored lower than Australian managers on Relaxation (Situational and Real). Both male managers and female managers from Canada scored significantly lower on Relaxation (Situational) than did their Australian counterparts. For senior executives and managers together, Canadians scored higher on Compliance (Situational) and lower on Relaxation (Real). Females scored higher on Sociability (both Situational and Real) and lower on Relaxation (both Situational and Real).
Hackett & Associates (2005b) reported WS results for 1654 entrepreneurs; 1509 from the U.S. and 145 from the U.K. Seventy-nine percent of the entrepreneurs were typed as WS Leaders. No nation differences were found in WS types or in WS profiles. With respect to the WS dimensions scores, males scored higher than females on Dominance (both Situational and Real); females scored higher on Sociability (both Situational and Real) and on Compliance (Real). U.K. males scored higher on Sociability- Situational than did U.S. males.
Research on The McQuaig Job Survey (JS)
The JS is a 21-item instrument that supervisors or other individuals most familiar with a particular job complete. It provides a description of the job in terms of WS profiles. Specifically, once the job profiling is done, individuals with WS profiles most closely matched to the job profile are considered to have the personality disposition most suitable to that particular job. Research suggests that the closer the match between a job profile and the WS profile of an individual, the higher the probability the individual will succeed in the job.
Each of the four dimensions of the WS (Dominance, Sociability, Relaxation and Compliance) is represented within each of the 21 items of the JS. The instrument is completed in the same manner as the WS. The statement which best describes the characteristics of a particular job is indicated by giving an “A” response, the next most characteristic statement is given a “B” response, and so forth.
The supporting research presented below is a summary of a technical report prepared by David Abbey (Evaluation Research Ltd., 1990; Nov.).
Scale reliabilities
Forced-choice tests are, by definition, ipastive. That is, the selection and ranking of alternatives within each item are not independent of one another. Accordingly, calculation of reliability coefficients in the usual manner (split-half or odd- even) may result in over-estimates of individual factor reliabilities. Accordingly, the split- half reliabilities reported for the four dimensions of the JS (N=67) presented below are probably somewhat inflated. They are compared with the split-half reliabilities for the same four dimensions of the WS (N=418).
Split-half reliabilities for factors of the JS and the WS
| Factor | JS | WS |
| D | .91 | .86 |
| S | .88 | .73 |
| R | .78 | .80 |
| C | .88 | .7 |
Two additional approaches were followed in assessing the internal reliabilities of the four factors of the JS. In the first, split-half reliabilities were calculated for the first choice only within each of the 21 items; in the second, the reliability of the last choice only was calculated. These reflect situations with the maximum and minimum degrees of freedom within items respectively. Additionally, the Spearman-Brown formula was used to correct for the splitting of the 21-item scale. These corrected reliabilities were:
| Factor | first choice only | last choice only |
| D | .90 | .88 |
| S | .75 | .63 |
| R | .88 | .71 |
| C | .79 | .76 |
These results suggest that the JS has acceptably high degrees of internal consistency across the four factors.
Profile reliabilities
Because of the ipsative nature of the JS, a more comprehensive measure of its reliability is recommended. Accordingly, two job profiles were calculated from each of 60 completed JS forms. The first was based on the odd- numbered items, the second based on the even numbered items. Each factor score was pro-rated so that the two profiles yielded a total of 168. The 120 resulting profiles were then translated into the appropriate profile category. A case-by-case comparison of odd and even profiles was then made. Of the 60 cases, 28 were exact matches. That is, when the odd items on the JS yielded a generalist profile, so did the even items; when the odd items yielded a specialist so too did the even items, and so forth. The below contingency Table summarises the matches and mismatches, after collapsing the individual profiles into Leader, Expert, and Transition. The two sources of profiles are significantly related (Chi- Square = 11.77, d.f. =1, p. < .0001). Because of the small expected values for Transitions, these three cases were omitted from the analysis.
Reliability of JS based on matching of profiles from both odd-numbered and even-numbered items
- Profiles from even-numbered items
- Profiles from odd-numbered items
| Profile | Leader | Expert | Transition | Total |
| Leader | 7 | 19 | 1 | 27 |
| Expert | 24 | 7 | 0 | 31 |
| Transition | 2 | 0 | 0 | 2 |
| Total | 33 | 26 | 1 | 6 |
Overall, the above research is supportive of the scale and profile reliabilities of the JS.
Construct validity
One approach to examining the construct validity of the JS would be to assess the correspondence between WS profiles of job incumbents and the JS profiles for their current positions. That is, for persons who are performing existing jobs, we would expect a correspondence between their WS scores and their JS scores, assuming that individuals seek out jobs for which they feel they are most suited.
The correlation between the two scores for each dimension can be regarded as a form of construct validity. As shown below, all four dimensions were significantly correlated between the two instruments.
| Factor | WS-JS Correlation | p. (signif. level) |
| D | .40 | .00 |
| R | .34 | .00 |
| S | .22 | .04 |
| C | .52 | .00 |
It is noteworthy that all four correlations are statistically significant though they are based on two completely different types of assessment. That is, the WS scores result from self-assessment. The JS scores are derived from a supervisor’s assessment of the demand characteristics of a particular job. In completing the JS, supervisors were specifically instructed to assess the job and not the incumbent. Accordingly, the fact that these factors are significantly correlated indicates that there is a significant relationship between the job characteristics and the temperament factors of the incumbents in these jobs.
Additionally, the match between JS and WS profiles on the same sample of job incumbents was assessed. The results are shown below.
WS Profiles
| JS profiles | Leader | Expert | Transition | Total |
| Leader | 32 | 9 | 0 | 41 |
| Expert | 8 | 15 | 1 | 24 |
| Transition | 2 | 0 | 0 | 2 |
| Totals | 42 | 24 | 1 | 67 |
If the two cases for which a Transition profile is indicated by the WS are deleted — since in reality this would likely result in a re-examination of the expectations of the job — then the chi-square for independence (Yate’s correction) is 12.32 (p. < .05, d.f. = 2). These results are consistent with the significant correlations between the dimension scores of the JS and WS. Together, this research supports the notion that the employees involved in these studies were in jobs which matched their temperaments.
Predictive validity
The true test of the validity of the JS is to determine whether individuals in positions for which JS scores match WS scores are higher performers than those in positions for which JS scores do not match their WS scores. Accordingly, supervisory ratings of performance were assessed against JS–WS matches/mismatches. The two-way contingency table presented below shows the relationship of performance (Lo-Hi) to profile matching.
Relationship of Employee Performance to the WS-JS Profile Matching
| WS-JS Match | Lo Perf. | Hi Perf. | Tot. |
| Matched | 19 | 28 | 47 |
| Not-matched | 13 | 7 | 20 |
| Totals | 31 | 35 | 67 |
Of the 47 persons whose profiles were matched on the two instruments, 28 (60%) had high performance levels. Of 20 persons whose profiles were not matched on the two instruments, only 7 (35%) had performance levels above the median. The phi coefficient corresponds to an increase of 22% in the success of predicting the level of performance rating from a knowledge of whether the two profiles were matched (Chi-Square = 3.48, d.f. =1, p. < .10).
Summary of JS validity analysis
This study of 67 job incumbents showed that the job profiles generated by the JS related significantly to the WS profiles of job incumbents. Furthermore, when the JS and the WS profiles matched, it was more likely that the incumbent would receive a higher performance rating than when the profiles did not match.
Download The Full Technical Manual
References
Allport, G.W. & Odbert, H.S. (1933). Trait names: a psycho-lexical study. Psychological Monographs, 47, 171-220 (1, Whole No. 211).
Anastasi, A. (1988). Psychological testing (Sixth Edition). New York: Macmillan Publishing Co.
Barrick, M.R., & Mount, M.K. (1991). The big five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26.
Cantor, N. & Mischel, W. (1979). Prototypes in person perception. In L. Berkowitz (Ed.), Advances in experimental social psychology, 12, 2-52, San Diego, CA: Academic Press.
Cattell, R.B. (1947). Confirmation and clarification of primary personality factors. Psychometrical, 12, 197-220.
Cattell, R.B., Eber, H.W., & Tatsuoka, M.M. (1970). Handbook for the Sixteen Personality Factor Questionnaire. Champaign, IL: Institute for Personality and Ability Testing.
Choynowski, M. (undated). To determine the relationship of the dominance factor to success in sales. Technical Report.
Costa, P.T. & McCrae, R.R. (1992). Revised NEO Personality Inventory (NEO PI-R) and NEO Five Factor Inventory (NEO-FFI). Professional Manual. Psychological Assessment Resources, Inc. P.O. Box 998, Odessa, Florida.
Eshelman & Townsend (1986; July). Use of the McQuaig Word Survey to predict performance and turnover of Bretton’s sales representatives. Technical Report.
Eshelman & Townsend (1986; Oct.). Use of the McQuaig Word Survey to predict sales performance of Curtis Industries representatives.
Eshelman & Townsend (1987; March). Use of the McQuaig Word Survey to predict performance of Kraft territory managers. Technical Manual.
Eshelman & Townsend (June, 1987a). The McQuaig Institute Zellers managerial study. Technical Report.
Eshelman & Townsend (June, 1987b). McQuaig Institute Prudential study. Technical Report.
Eshelman & Townsend (1987; August). The McQuaig Institute Bretton’s performance of salespeople. Technical Report.
Eshelman & Townsend (1988; Aug.). Performance of Miracle managers: Food-Mart and Ultra-Mart stores. Technical Report.
Eshelman & Townsend (1990; Jan.). Performance of Sainsbury store managers. Technical Report.
Eshelman & Townsend (1991; Sept.). Kmart assistant manager study. Technical Report.
Eshelman & Townsend (1992; Feb.). Kmart operational managers study. Technical Report.
Eshelman & Townsend (1992; March). Friden Alcatel sales applicant study.
Evaluation Research Limited (1987; Nov.). Bretton’s Word Survey data – Statistical analysis II. Technical Report.
Evaluation Research Limited (1990; Nov.). The MJSII (The McQuaig Job Survey®) – Revised: Development, reliability and validity studies. Technical Report.
Evaluation Research Limited (1991; Jan.). Reliability of the McQuaig Word Survey: Technical Report.
Evaluation Research Limited (1991; Oct.). Context study. Technical Report.
Fiske, D.W. (1949). Consistency of the factorial structures of personality ratings from different sources. Journal of Abnormal and Social Psychology, 44, 329-344.
Goldberg, L.R. (1993). The structure of phenotypic personality traits. American Psychologist, 48, No. 1., 26-34.
Guilford, J.P. (1986). When not to factor analyse. In D. Jackson, & S. Messick (Eds.)
Problems in Human Assessment, New York: McGraw-Hill, p. 309-317. Guion, R.M. (1965). Personnel testing. New York: McGraw Hill Book Co.
Hackett & Associates Human Resources Consultants Inc. (1994; June). The McQuaig Word Survey: Analysis by sex and race with a U.S. student sample. Technical Report.
Hackett & Associates Human Resources Consultants Inc. (1996; May). Relationships among the McQuaig Word Survey (WS), The McQuaig Situation Survey (MSS), the Myers Briggs Type Indicator (MBTI) and the NEO-Five Factor Inventory (NEO-FFI): An exploratory study.
Hackett & Associates Human Resources Consultants Inc. (2000, November), Young Entrepreneurs Organisation Study. Technical Report.
Hackett & Associates Human Resources Consultants Inc. (2001;April). A Cross-Cultural Study of the McQuaig Word Survey: Canada, China and India. Technical Report.
Hackett & Associates Human Resources Consultants Inc. (2005a). Australian and Canadian business leaders: A combined and comparative analysis of WS types, profiles and dimension scores.
Hackett & Associates Human Resources Consultants Inc. (2005b). U.S. and U.K. entrepreneurs: A combined and comparative analysis of WS types, profiles and dimension scores.
Hampson, S.E., John, O.P., & Goldberg, L.R. (1986). Category breadth and hierarchical structure in personality: Studies in asymmetries in judgments of trait implications. Journal of Personality and Social Psychology, 51, 37-54.
Hogan, R. (1991). Personality and personality measurement. In M.D. Dunnette & L.M. Hough (Eds.), Handbook of industrial and organisational psychology (2nd ed., 2, 873- 919). Palo Alto, CA: Consulting Psychologists Press.
[/su_column]
Holland, J.L. (1973). Making vocational choices: A theory of careers. Englewood Cliffs, N.J.: Prentice- Hall.
John, O.P. Hampson, S.E., & Goldberg, L.R. (1991). The basic level in personality-trait hierarchies: Studies of trait use and accessibility in different contexts. Journal of Personality and Social Psychology, 60, 348-361.
Levin, J. (1977). Elementary statistics in social research (2nd. Ed.). New York: Harper & Row, Publishers.
Lewis, C. (1991; Dec.). Report on the use of the McQuaig Word Survey and The McQuaig Occupational Test Within Barclays Financial Services, U.K.. Technical Report.
McAllister, L.W. (1986). A practical guide to CPI interpretation. Palo Alto, CA: Consulting Psychological Press.
Mieczyslaw, C. (Undated). To determine the relationship of the Dominance factor to success in sales. Technical Report.
Miller, Ginsburg & Brien (1979; Oct.). Statistical findings. Technical Report.
Miller, Ginsburg & Brien (1980; Jan.). A comparative study of high versus low insurance producers on the McQuaig Word Survey. Technical Report.
Miller, Ginsburg & Brien (1980; Feb.). Test-retest reliability. Technical Report.
Miller, Ginsburg & Brien (1980; March). Validation study of the Word Survey. Technical Report.
Miller, Ginsburg & Brien (1980a; April). Characteristics of effective versus ineffective Sun Life insurance agents on the McQuaig Word Survey. Technical Report.
Miller, Ginsburg & Brien (1980b; April). Characteristics of effective versus ineffective transit workers on the McQuaig Word Survey. Technical Report.
Miller, Ginsburg & Brien (1980; June). Personality types measured by the McQuaig Word Survey. Technical Report.
Miller, Ginsburg & Brien (1981; April). Correlations of Word Survey scales with other psychological tests. Technical Report.
Nishisato, S. (undated). Predictive validity of McQuaig’s “The Word Survey”. Technical Report.
Parker Allen Company (undated). The Word Survey performance of males and females. Technical Report.
Parker Allen Company (undated). Construct validity of the Word Survey using the 16 Personality Factor Test (16PF). Technical Report.
Parker Allen Company (1981; June). Performance of above average and below average railroad reservation clerks on the McQuaig Word Survey. Technical Report.
Parker Allen Company (1981; Sept.). Predicting performance and turnover of Investor Syndicate agents. Technical Report.
Parker Allen Company (Oct. 1982a). Use of the McQuaig Word Survey to predict new agent Performance. Technical Report.
Parker Allen Company (Oct. 1982b). Use of the McQuaig Word Survey to predict business reporter turnover and performance. Technical Report.
Parker Allen Company (1982; Dec.). Use of the McQuaig Word Survey to predict productivity and longevity of Sun Life insurance agents: A tracking study of new agents. Technical Report.
Parker Allen Company (1983; Jan.). Use of the McQuaig Word Survey to predict performance of Amdahl service field engineers. Technical Report.
Parker Allen Company (1983; May). The Word Survey performance of males and females. Technical Report.
Parker Allen Company (1983; July). Use of the French and English versions of the McQuaig Word Survey. Technical Report.
Parker Allen Company (1983; Aug.). Update on the reliability of factor scores and profile types derived from the McQuaig Word Survey. Technical Report.
Parker Allen Company (1983; Sept.). Use of the McQuaig Word Survey to predict performance of Jamaica agents. Technical Report.
Parker Allen Company (1984; March). The McQuaig Word Survey CEO president study. Technical Report.
Parker Allen Company (1984; May). Use of the McQuaig Word Survey to predict performance of new agents at Montreal Trust: Preliminary report of 12 month data. Technical Report.
Parker Allen Company (1984; Oct.). Use of the McQuaig Word Survey to predict performance of Imperial Life agents: Preliminary 6-month analyses. Technical Report.Parker Allen Company (1985; Feb.). Use of the McQuaig Word Survey to predict performance and tenure of work wear routemen: Preliminary analysis. Technical Report.
Parker Allen Company (1985; March). Use of the McQuaig Word Survey to predict performance of Prudential Assurance agents. Technical Report.
Shizuhiko, N. (Undated). To investigate the predictive validity of the Word Survey in discriminating between successful and unsuccessful managers. Technical Report.
Tett, R.P., Jackson, D.N. & Rothstein, M. (1991). Personality measures as predictors of job performance: A meta-analytic review. Personnel Psychology, 44, 703-742.
Footnotes
- A personality type is defined as a cluster of personal attributes that best describe an individual
- These have been referred to as the “big five” personality factors.
- Goldberg (1993) notes with regard to the “big five” that these broad domains incorporate hundreds, if not thousands, of traits. Factor 1 (Urgency or Extraversion) contrasts such traits as talkativeness, assertiveness, and activity level with traits such as silence, passivity, and reserve; Factor 2 (Agreeableness or Pleasantness) contrasts traits such as kindness, trust, and warmth with such traits as hostility, selfishness and distrust; Factor 3 (Conscientiousness or Dependability) contrasts such traits as organisation, thoroughness, and reliability with traits such as carelessness, negligence, and unreliability; Factor 4 (Emotional Stability vs. Neuroticism) includes such traits as nervousness, moodiness, and temperamentality; and Factor 5 (Openness to Experience) contrasts such traits as imagination, curiosity, and creativity with traits such as shallowness and imperceptiveness” (p. 27). The point is that each of these broad high level factors are really quite heterogeneous in nature. The four factors of the WS were tailored to capture select key components of the “big five”.
- For supporting research on the reliability and validity of The McQuaig Word Survey®, see Evaluation Research Limited, 1990; Nov.).
- In reviewing the research summarised in this technical manual, it should be understood that some of the names of the profile types have changed over the years, though the profiles themselves have remained exactly the same. Specifically, “Nice Guy or Gal” was changed to “Cooperator”, “Friendly Pusher” was changed to “Enthusiast”, “Autocrat” was changed to “Pioneer”, “Conflict” was changed to “Transition” and “Researcher” was changed to “Administrator”. To avoid confusion in reviewing the earlier research on the WS, the most recent names given to the profile types have been used throughout this manual.
- Research has shown that different forms of the Survey, defined by combinations of the four terms (tetrads) differing from the original McQuaig Word Survey®, do not make any difference in respondent scores achieved over the four factors (D, S, R, C). Moreover, the layout and the arrangements of the words on the WS do not affect the relationships among factor scores nor do they affect the profiles of persons completing the WS. The context – the particular four words that are clustered together among which choices are made, is not a factor in determining an individual’s overall score or profile. See Miller, Ginsburg & Brien (1980; Feb.).
- The layout of the WS is such that within each block of four sets of words, each factor (D, S, R, C) appears once and only once in each position. The pattern is then repeated for each block of four sets. To control for possible order and position effects the odd numbered blocks (1 and 3) were used for the first (Odd) profile for each individual, the even numbered blocks (2 and 4) were used to produce the second (Even) profile. These four blocks of four sets each utilised 16 of the 21 sets of the Survey. Sets 17, 18 and 19 were added to the Odd blocks, sets 20 and 21 were added to the Even blocks. The differential in numbers of sets was adjusted by weighting totals prior to determining the overall factor scores and determining each individual’s profile.
- Holding back is a profile that drops at least 10 points on the Dominance scale from the Situational to the Real side of the survey and concomitantly rises 10 or more points on the Relaxed (R) or Compliance (C) scale from real to Situational sides. A transition classification is used when the Dominance and C are equal or very close together on either both the real or Situational sides of the instrument.
- A “persuader” has the following profile: high D, high S, Low R, Low C with the S at least five points higher than the D. The profile is typical of many good salespeople, supervisors and S/he would be best in an environment where public relations and interpersonal skills are the keys to getting results.
RICK D. HACKETT
Professor of Human Resource Management & Canada Research Chair of Organisational Behaviour and Human Performance
Michael G. DeGroote School of Business, McMaster University
Dr. Hackett received his Ph.D in industrial-organisational psychology in 1985 from Bowling Green State University (Ohio) and is a Professor of Human Resources Management at the Michael G. DeGroote School of Business and President of Hackett & Associates Human Resources Consultants Inc.
He is also Past-President of the Canadian Society for Industrial-Organisational Psychology (CSIOP) of the Canadian Psychological Association, a professional association of approximately 170 Canadian industrial-organisational psychologists.
In June, 1997, he was elected “Fellow” of the Canadian Psychological Association.
Additionally, Dr. Hackett is an ad hoc reviewer for several scholarly journals in management and was appointed Editor of the Canadian Journal of Administrative Sciences (Human Resources Division) for a six-year term effective January 1, 1997.
Dr. Hackett was Guest Editor of a special issue of Canadian Psychology, published in 1998, and was elected a member of International Who’s Who of Professionals in October 1997.
He has also co-authored a book titled: “Recruitment and Selection in Canada”, published by ITP Nelson (2005).
TABLE 1
Overview of Norms and Profile Distribution
| STUDY | SAMPLE | N | GENERAL FINDINGS |
The McQuaig Institute. Reviewed and verified by Hackett & Associates, Human Resources Consultants Inc. (February, 2012) | Global sample, various positions | 352,613 | Norms * “Real” Side Mean Factor scores: Dominance = 47, Sociability = 47, Relaxation = 33, Compliance = 43 * “Situational” Side Mean Factor scores: Dominance = 44, Sociability = 48, Relaxation = 34, Compliance = 41 |
The McQuaig Institute. Reviewed and verified by Hackett & Associates, Human Resources Consultants Inc. (February, 2012) | Global sample, various positions | 352,613 | Word Survey Profile Type Distribution * Generalist: 20%; Enthusiast: 16%; Pioneer: 14%; Specialist: 14%; Persuader 11%; * Transition: 9%; Cooperator: 6%; Possible Transition*: 6%; Administrator: 4% |
The McQuaig Institute. Reviewed and verified by Hackett & Associates, Human Resources Consultants Inc. (February, 2012) | Global sample, various positions | 352,613 | Word Survey Distribution based on Situation Adjustments It was found that 28% of the Generalist Group was holding back while 4% were stretching; 15% of the Specialist Group was holding back while 10% were stretching; 33% of the Transition Group was holding back while 8% were stretching. |
TABLE 1
Overview of Reliability Studies
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
Miller, Ginsburg, & Brien (June 1980) | Managers | 418 | Internal Consistency | * “Real” Side Correlations: Dominance = .87, Sociability = .76, Relaxation = .78, Compliance = .76 * “Situational” Side Correlations: Dominance = .86, Sociability = .73, Relaxation = .80, Compliance = .76 |
Miller, Ginsburg, & Brien (Feb. 1980) | Unknown | 231 | Test-Retest (Time Span: Several Months) | * “ Real” Side Correlations: Dominance = .74, Sociability = .57, Relaxation = .72, Compliance = .66 * “Situational” Side Correlations: Dominance = .75, Sociability = .76, Relaxation = .66, Compliance = .68 |
| Parker Allen Co. (Aug. 1983) | Various Administra- tors | 121 | Test-Retest (Time Span: Several Months) | * “Real” Side Correlations: Dominance = .81, Sociability = .71, Relaxation = .78, Compliance = .82 * “Situational” Side Correlations: Dominance = .81, Sociability = .84, Relaxation = .77, Compliance = .80 |
| Parker Allen Co. (Aug. 1983) | Various Administra- tors | 121 | Test-Retest (Time Span: Several Months) | * Reliability across the six profile types: Generalist, Pioneer, Specialist, Enthusiast, Cooperator, & administrator is .65 (Cramer’s V Statistic) * When the six profile types are reclassified into “profile families”, reliability is .82 (Cramer’s V) |
| Evaluation Research Ltd. (Jan. 1991) | Various Job Applicants | 208 | Parallel Forms | * Form A (Situational): 72% were accurately matched according to family * Form A (Real): 73% were accurately matched according to family * Form B (Situational): 67% were accurately matched according to family * Form B (Real): 73% were accurately matched according to family |
TABLE 2
Overview of Construct Validity Studies
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
| Eshelman & Townsend (1983) | University Students (M.B.A.) | 63 | Sixteen Personality Factor Test (16PF) | Significant “Real” Side Correlations for: * WS Dominance – Dominance .45, Rebelliousness .34 * WS Sociability – I+++++++++++++++++++++++mpulsivity .59, Risk Taking .58, Sociability .42, Self Sufficiency -.27 * WS Relaxation – Risk Taking -.61, Impulsivity -.48, Sociability -.48, Self Sufficiency .27, Intelligence .27 * WS Compliance – Risk Taking -.52, Impulsivity -.50, Dominance -.34, Rebelliousness -.28, Insecurity .32 * A very similar pattern of results emerged on the “Situational” side. |
| Miller, Ginsburg, & Brien (April, 1981) | University Students (M.B.A.) | 78 | California Psychological Inventory (CPI) & Rathus Assertiveness Scale (RAS) | Significant “Real” Side Correlations for: * WS Dominance – Dominance .40, Capacity for Status .40, Sociability .33, Social Presence .37, Psychological Mindedness .30, RAS .52 * WS Sociability – Dominance .37, Sociability .49, Social Presence .40, Self- Acceptance .41, Self-Control -.33, RAS .42 * WS Relaxation – Dominance -.50, Sociability -.45, Social Presence -.40, Self- Acceptance -.43, Self-Control .22, RAS -.56 * WS Compliance – Dominance -.41, Capacity for Status -.34, Sociability -.50, Social Presence -.48, Self-Acceptance -.34, Responsibility .30, Femininity .27, RAS -.56 * A very similar pattern of results emerged on the “Situational” side. |
| Hackett & Associates, Human Resources Consultants Inc. (July 1996) | University Students (3rd & 4th year Commerce) | 206 | NEO Five Factor Inventory (NEO-FFI) | Significant “Real” Side Correlations with NEO-FFI: * WS Dominance – Agreeableness -.40 * WS Sociability – Extraversion .58 * WS Relaxation – Extraversion -.43 * WS Compliance – Agreeableness .34 WS Profile Types can be differentiated based on their overall patterns of scores on the NEO-FFI: * WS Leaders are less “neurotic” and less “agreeable” than WS Experts. * Cooperators are more “neurotic” than Persuaders; Specialists are more “introverted” than all other profile types except Pioneers; Cooperators are more “agreeable” than Generalists and Pioneers. |
TABLE 2 (Continued) Overview of Construct Validity Studies
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
| Hackett & Associates, Human Resources Consultants Inc. (July 1996) | University Students (3rd & 4th year Commerce) | 206 | Myers-Briggs Type Indicator (MBTI) | * Most WS Leaders are classified as ISTJs (serious, quiet, thorough, logical, realistic, dependable, persevering, and independent minded). * Most WS Experts are classifies as ISFJs (quiet, friendly, responsible, conscientious, stable, thorough, accurate, non-technical, patient with details, loyal, and considerate of others). |
| Miller, Ginsburg, & Brien (March, 1980) | Employees of 14 Different Corpora- tions | 200 | Observable Behaviours (Supervisory Checklist) | * WS profile patterns “map onto” or reflect behaviours across six areas (verbal communication, ability to work under pressure, level of autonomy, interpersonal skills, work habits, achievement orientation). |
TABLE 3
Overview of Criterion Validity Studies with Sales Occupations
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
Miller, Ginsburg, & Brien (April 1980) | Life Insurance Agents | 184 | Supervisory Performance Ratings | * Profile type is significantly related to agents’ performance levels (Validity Coefficient = .44) * Generalists/Pioneers are the best performers; Transition/Holding Back types are the poorest performers * Generalists/Pioneers are the least likely to be terminated whereas Transition/Holding Back types are the most likely to be terminated * Better performers tend to have slightly higher Sociability scores |
Miller, Ginsburg, & Brien (Jan. 1980) | Life Insurance Agents | 97 | Annual Sales | * Top performers have higher Dominance scores and lower Sociability and Compliance scores on the “Situational” side of the WS than low performers * Top performers have higher Dominance scores on the “Real” side than low performers |
| Parker Allen Co. (Dec. 1982) | Life Insurance Agents | 184 | Turnover After 3 Years | * Transition/Holding Back types are most likely to turnover (89%) * Generalists/Pioneers are least likely to turnover (38%) * Other WS types are in between (50%) |
| Parker Allen Co. (Oct. 1984) | Life Insurance Agents | 62 | Sales Performance (Commissions) | * Generalists/Pioneers are more likely to earn high commissions than other profile types |
| Choynowski (Undated) | Various Salesmen | 187 | Managerial Performance Ratings | * Higher performers obtain higher Dominance scores and lower Relaxation and Compliance scores on the “Real” side of the WS |
| Parker Allen Co. (Sept. 1981) | Investment Firm Agents | 249 | Supervisory Performance Ratings and Turnover | * Generalists/Pioneers are the best performers; Transition/Holding Back types are the poorest performers * Transition/Holding Back types are more likely to be dismissed than Generalist/Pioneers * WS type contributes significant incremental variance to the prediction of performance (over the prediction provided by 8 other predictors) |
TABLE 3 (Continued) Overview of Criterion Validity Studies with Sales Occupations
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
| Parker Allen Co. (Oct. 1982) | Real Estate Agents | 139 | Sales Performance | * Generalists/Pioneers outperform Enthusiasts/Specialists and Transition/Low Morale types and are least likely to turnover * Transition/Holding Back types are the poorest performers |
| Parker Allen Co. (May 1984) | Real Estate Agents | 139 | Sales Performance and Turnover | * Generalists/Pioneers outperform Enthusiasts/Specialists and Transition/Low Morale types Transition/Holding Back types are the poorest performers and have the highest turnover rate |
| Eshelman & Townsend (July 1986) | Retail Sales Representa- tives | 150 | Performance Ratings | * The poorest performers are the Specialists/Cooperators * The poorest performers have higher than average Compliance scores on the “Situational” side of the WS * Good performers have higher Dominance scores and lower Compliance scores on the “Real” side of the WS than average or poor performers |
| Eshelman & Townsend (Aug. 1987) | Retail Sales Representa- tives | 345 | Performance Ratings | * Generalists and Enthusiasts are the best performers * Cooperators are the poorest performers |
| Evaluation Research Ltd. (Nov. 1987) | Retail Sales Representa- tives | 321 | Turnover | * Leaders (Generalists, Persuaders & Pioneers) have higher turnover rates than Experts (Specialists, Cooperators & Enthusiasts * Experts who have low Relaxation scores tend to be the best performers |
| Curtis Industries (Oct. 1986) | Sales Representa- tives | 55 | Sales Performance | * Enthusiasts are the best performers, followed by Generalists and then Specialists * Holding Back types are the poorest performers * Poor performers have higher Dominance scores and lower Compliance scores than average and high performers |
| Parker Allen Co. (Sept. 1983) | Life Insurance Agents | 62 | Sales Performance | * Generalists/Pioneers are the highest performers * Transition/Holding Back types are the lowest performers * Specialists/Cooperators/Enthusiasts fall between the other types in their performance |
TABLE 3 (Continued) Overview of Criterion Validity Studies with Sales Occupations
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
| Parker Allen Co. (March 1985) | Life Insurance Agents | 117 | Performance Ratings | * The correspondence between predicted and actual performance is indicated by a 2/3rds “hit rate” |
| Eshelman & Townsend (June 1987) | Life Insurance Agents | 287 | Performance Ratings | * Cooperators, Generalists & Specialists are the best performers * Enthusiasts are the poorest performers |
| Lewis (Dec. 1991) | Financial Services Sales | 244 | Sales Performance | * The factor scores on the WS permit the identification of good vs. poor performers * There is an overall performance benefit of approximately 11% to be gained from using the WS as a selection instrument |
| Eshelman & Townsend (March 1987) | Kraft Territory Managers (Sales) | 217 | Performance Ratings | * Generalists and Pioneers outperform all other profile types * High performers have higher Dominance scores and lower Relaxation and Compliance scores than lower performers |
| Hackett & Associates Human Resources Consultants Inc. (August 1995) | RBC Security Agents | 134 | Total Assets | * WS “Real C” correlated with total assets (r=.30, p < .05). * Those with “Real C” scores of 18 or less realised mean assets of $3,720,331; * Those with “Real C” scores between 19-30 realised mean assets of $5,561,063; * Those with “Real C” scores of 31 or more realised mean assets of $7,637,156.50 |
TABLE 4
Overview of Criterion Validity Studies with Managers
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
| Nishisato (Undated) | Various Managers | 102 | Performance Ratings | * Managers’ performance levels were predicted 92.9% of the time based on their WS factor scores * Successful managers obtain higher scores on Dominance and Sociability and lower scores on Relaxation and Compliance than unsuccessful managers |
| Eshelman & Townsend (Aug. 1988) | Grocery Chain Managers | 75 | Performance Ratings | * Pioneers are the best performers * Successful managers have higher than average Dominance scores and lower than average Relaxation and Compliance scores on the “Situational” side of the WS * On the “Real” side, strong performance is associated with higher than average Dominance Scores and lower than average Compliance scores |
| Eshelman & Townsend (Sept. 1991) | Kmart Assistant Store Managers | 188 | Performance Ratings | * Generalists, followed by Pioneers, are the highest performers * High performance is associated with higher Dominance scores, satisfactory performance is associated with higher Sociability scores, and poor performance is associated with higher Relaxation and Compliance scores |
| Eshelman & Townsend (Feb. 1992) | Kmart Operational Managers | 127 | Performance Ratings | * Pioneers and Generalists are the highest performers * Better performers have higher Dominance scores * Higher Relaxation and Compliance scores are associated with poorer performance |
| Eshelman & Townsend (June 1987) | Zellers Store Managers | 86 | Performance Ratings | * Pioneers are the highest performers * The best performers achieve higher Dominance and lower Relaxation scores than the average or lowest performers |
| Parker Allen Co. (March 1984) | Corporate Presidents & CEOs | 75 | Attainment of Presidency/CEO Status | * Over 90% of CEOs/Presidents are Pioneers (41.3%), Generalists (37.3%), or Administrators (16%) |
TABLE 4 (Continued) Overview of Criterion Validity Studies with Managers
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
| Eshelman & Townsend (Jan. 1990) | Sainsbury Store Managers | 39 | Performance Ratings | * Pioneers and Administrators are most likely to be top performers * Specialists and Holding Back types are most likely to be the poorest performers * Top performers achieve higher Dominance scores than poor performers * Poor performers have higher Sociability and Relaxation scores than top performers on the “Situational” side of the WS and higher Compliance scores on the “Real” side |
TABLE 5
Overview of Criterion Validity Studies with Non Sales/Non Managerial Samples
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
Miller, Ginsburg & Brien (April, 1980) | Transit Workers | 25 | Performance Ratings | * High Relaxation scores and low Sociability scores are associated with higher performance |
| Parker Allen Co. (Oct. 1982) | Business Reporters | 180 | Performance & Turnover | * Generalists and Pioneers are least likely to turnover * Holding Backs and Specialists/Enthusiasts/Cooperators are most likely to turnover * WS profile is unrelated to job performance |
| Parker Allen Co. (Jan. 1983) | Field & Service Engineers | 54 | Performance Ratings | * Among Service Engineers the highest performers are Cooperators, Specialists and Administrators * The best performing Service Engineers have higher Relaxation scores and lower Dominance scores than poorer performers * Among Field Engineers, the highest performers are Administrators, Specialists, and Transitions * The best performing Field Engineers have higher Relaxation scores and lower Dominance and Sociability scores than poor performers * For Field and Service Engineers combined, Dominance correlates negatively and Relaxation correlates positively with performance |
| Parker Allen Co. (Feb. 1985) | Work Wear Routemen | 89 | Performance Ratings and Turnover | * Generalists, Pioneers, Administrators and Persuaders are the best performers * Transition/Holding Back types have the highest involuntary dismissal rates whereas Enthusiasts/Specialists/Cooperators have the highest voluntary resignation rates |
| Parker Allen Co. (June 1981) | Railroad Reservation Clerks | 86 | Past & Present Supervisory Performance Ratings | * Present performance ratings are unrelated to WS profile type * Generalists receive the highest past performance ratings, then Specialist, then Enthusiasts and, finally, Transitions receive the lowest past ratings * Relaxation is related positively with performance whereas Sociability is related negatively |
| Hackett & Associates (2009) | Claims | 105 | Performance & Potential ratings Ratings | * Among claims representatives the Specialist and the Enthusiast received the highest performance ratings * The Good category has a higher percentage of Expert profiles (59%) than Leader profiles (33%) * The Superior category also has a higher percentage of Expert profiles (54%) than Leader profiles (32%) * Lateral Potentials have a greater percentage of Expert profiles (55%) than Leader profiles (38%) * Promotion Potentials have a greater percentage of Expert profiles (65%) than Leader profiles (26%). c2 (1, 23) = 3. 52, p < 06. |
| Hackett & Associates (2009) | Underwriting | 89 | Performance & Potential ratings Ratings | * The Specialist is the most common profile in the Good (35%) and Superior (30%) groups, c2 (7, 129) = 16.5, p < .05 * The Superior category has a higher percentage of Leader profiles (47%) than Expert profiles (40%) while the Good category has a higher percentage of Expert profiles (68%) than Leader profiles (24%), c2 (1, 116) = 8.8, p < .01 * Lateral Potentials have a greater percentage of Expert profiles (71%) than Leader profiles (24%), while more individuals with Promotion Potential have Leader profiles (46%) than Expert profiles (36%), c2 (1, 118) = 10.5, p < .001 |
TABLE 6
Overview of Criterion Validity Studies with Entrepreneurs
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS |
| Hackett & Associates HRC. Inc. (Nov. 2000) | Young Entrepreneurs Organisation | 1279 males 239 females | Revenues of $1,000,000 or more | * 79% were WS Leaders; 80.2% of the males and 70.7% of the females. * 60% of sample was comprised of Generalists (31.1%) and Pioneers (28.9%) * Most common among females were Generalists (32.2%) and Persuaders (20.9%) * WS dimension profile for total sample one of high Dominance, next highest Sociability, with substantially lower levels of Relaxation and Compliance * Females were similarly high in Dominance and Sociability; Males were higher on Dominance than on Sociability |
TABLE 7
Overview of Test Fairness Studies
| STUDY | SAMPLE | N | CRITERION | GENERAL FINDINGS | |||||||
| Parker Allen Co. (May 1983) | Male & Female MBA | 40 28 | Mean WS Factor Scores by Sex | * There are no differences between males and females except that females have slightly higher Sociability scores on the “Real” side of the WS * Males and females are equally distributed across profile types | |||||||
| Students | |||||||||||
| Parker Allen Co. (July 1983) | Male & Female | 73 131 | Mean WS Factor Scores | * There are no differences between males and females on distribution across WS profile types nor on mean factor scores | |||||||
| Students | by Sex | ||||||||||
| Parker Allen Co. | Male & | 64 | WS Profile Type | * There are no differences in WS profile types associated with either sex or age | |||||||
| (Oct. 1982a) | Female | 75 | by Sex and Age | ||||||||
| Sales | |||||||||||
| Agents | |||||||||||
| Parker Allen Co. | Male & | 126 | WS Profile Type | * There are no differences in WS profile types associated with sex | |||||||
| (Oct. 1982b) | Female | 54 | by Sex | ||||||||
| Business | |||||||||||
| Reporters | |||||||||||
| Eshelman & Townsend | Male & Female | 178 39 | WS Profile Type by Sex, Education, | * There are no differences in WS profile types associated with sex, education, age, or race | |||||||
| (March 1987) | Managers | Age and Race | |||||||||
| Eshelman & Townsend (Sept. 1991) | Kmart Assistant Managers (Mixed Sample) | 188 | WS Profile Type by Sex, Race, Age, Language, Education and Marital Status | * There are no differences in WS profile types associated with sex, race, age, language, education, or marital status | |||||||
| Lewis (1991) | Financial Services Sales | 244 | WS Profile Type by Sex | * There are no differences in WS profile types associated with sex | |||||||
| Eshelman & Townsend (March 1992) | Applicants for Sales Positions | 1,813 | WS Profile Type by Sex and Race | * There are no differences in WS profile types associated with sex or race | |||||||
| Parker Allen Co. (July 1983) | McGill University Students | 204 | WS Profile Type and Factor Scores by Language Version of WS English (n = 105) French (n = 99) | * There are no differences in WS profile types associated with the language version of the WS administered (English vs. French) * Francophones are more compliant and sociable and less relaxed than Anglophones | |||||||
| Hackett & Associates HRC Inc. (June, 1994) | U.S. University Students | 267 | WS Profile Type and Factor Scores by Sex and Race | * There are no differences in WS profile types associated with sex or race. * Higher Dominance Scores (Real & Situational) for males * Higher Compliance Scores for females (Situational) * Higher Sociability Scores (Real) for Whites over Mexican Americans * Higher Dominance Scores (Real) for African Americans over Whites | |||||||
| Hackett & Associates HRC Inc. (May, 1996). | U.S. University Students (Business) | 516 | WS Profile Types and factor scores by language version of WS, Sex, and Race English (n=292) Spanish (n=224) | · There are fewer Generalists, fewer Leaders and more Transitions among Hispanics writing the Spanish version than among the whites, African Americans or Mexican Americans writing the English version. · On WS-Situational, Hispanics writing the Spanish version have lower Dominance scores than African Americans, lower Sociability scores than Whites, and higher Compliance scores than African Americans or Whites. Whites have higher Sociability scores than Mexican-Americans. · On WS-Real, African Americans have higher dominance scores than Whites or Hispanics, lower Relaxation scores than Hispanics, and lower Compliance scores than Hispanics · There are no differences in WS profile types nor in WS Factors associated with sex on either language version. | |||||||
| Young Entrepreneurs Organization (YEO) – Hackett & Associates HRC. Inc. (Nov., 2000). | Male & Female Entrepreneurs (38 years of age or younger) | 1279 males 239 females | WS Profile Types and Factor Scores by Sex | · Approximately 79% of YEO members were typed as WS-Leaders; 80.2% Males, 70.7% females. More females (18.4%) than males (11.1%) were typed as Experts. · The most common WS profile type for males was Pioneer (31.2%) and Generalist (30.8%) whereas for females it was Generalist (32.2%) and Persuader (20.9%). · Higher Dominance scores (Real and Situational) for males · Higher Sociability scores (Real and Situational) for females | |||||||
| Hackett & | 350 university | 105 | WS profiles and | · No differences in WS responses between Indian males and Indian females | |||||||
| Associates HRC Inc. (2001; April) | business students; Canada, | Canada 136 China | WS types by Country and by Gender | · No differences in responses between Chinese males and Chinese females · No differences in responses between males and females across three countries · No cross-country differences in WS profiles or WS types | |||||||
| China & India | 109 | ||||||||||
| India | |||||||||||
Hackett & Associates HRC Inc. (2005; September) | 1654 Entrepreneurs | 1509 U.S. (YEO) 145 U.K. (Business Link) | WS Profile Types and Factor Scores by Gender and by Nation | · Approximately 79% of this group was typed as Leaders · No nation difference in WS types and profiles · Males scored higher on Dominance (both Situational and Real) and Relaxation (both Situational and Real) · Females scored higher on Sociability (both Situational and Real) and on Real Compliance · Males from the U.K. scored higher on Situational Sociability than did males from the U.S. | |||||||
Hackett & Associates HRC Inc. (2005; September) | 763 business leaders | 152 Canada 611 Australia 635 senior executives 128 managers 679 males 84 females | WS Profile Types and Factor Scores by Gender, by Nation and by senior executives/ managers | · Most typed as WS Leaders, followed by Transitions or Experts for the total sample and in each separate sample · No gender and nation difference in distribution of WS types and profiles in the total sample and in each separate sample · Greater representation of Leaders among senior executives (77%) than among managers (60%) · Greater percentages of Experts and Transitions among managers than among senior executives (19% versus 13% and 21% versus 10%, respectively). · Highest in Dominance, next highest in Sociability, with substantially lower levels of Relaxation and Compliance for the total sample and in each separate sample · Canadian (and male Canadian) senior executives scored higher than Australian (and male Australian) senior executives on Compliance (Situational) · Female senior executives scored higher than males on Sociability (Situational and Real) and lower on Relaxation (Situational) · Canadian managers scored lower than Australian managers on Relaxation (Situational and Real) · Both male managers and female managers from Canada scored significantly lower on Relaxation (Situational) than did their Australian counterparts. · For senior executives and managers together, Canadians scored significantly higher on Compliance (Situational) and lower on Relaxation (Real) · Females scored significantly higher on Sociability (both Situational and Real) and lower on Relaxation (both Situational and Real) · Senior executives scored higher than managers on Dominance (both Situational and Real) and lower on Compliance (both Situational and Real) | |||||||
Download: THE MCQUAIG WORD SURVEY®: Technical Manual – Charts and Graphs
©Copyright 1994-201O The McQuaig Institute® of Executive Development Ltd. Toronto Canada. All rights reserved. including the right to reproduce this report or any part thereof in any manner. ™Registered Trade Mark of the McQuaig Institute of Executive Development
Prepared by: Hackett & Associates Human Resources Consultants Inc. for the McQuaig Institute·® www.McQuaig.com
