Assessment of Abilities
Thomas F. Harrington
Source: Harrington, T. (1995). Assessment of abilities (ERIC Digest). Greensboro, NC: ERIC Clearinghouse on Counseling and Student Services. [ED389960]
Return to: | Readings in Educational Psychology | Educational Psychology Interactive |
This digest recommends assessing all of a person's abilities, not just some. It also discusses self-report in the context of ability assessment. The term aptitude often is used also in defining ability, and sometimes these terms are used interchangeably. Ability as used here follows Anastasi's (1988) concept of "developed abilities." Her viewpoint is that "all ability tests - whether they be designed as general intelligence tests, multiple aptitude batteries, special aptitude tests, or achievement tests - measure the level of development attained by the individual in one or more abilities" (p.413).
What Are The Major Abilities?
In 1976, Harrington and O'Shea identified 14 abilities found in U.S. Department of Labor publications and began assessing them in a self-report format. They reviewed 113 concurrent validity studies composed of vocational/technical programs, college and university majors, and employees in different jobs, and concluded that a high degree of agreement existed between the participants' self-reports on the 14 abilities and job analysts' findings of abilities required for job performance. Later, in 1992, Harrington altered the listing by adding organization and reading ability and collapsing computational with mathematical (Harrington & O'Shea, 1993). The 15 major abilities thus identified were:
|
|
Technical ability is a broader term that integrates many mechanical abilities. Retitling this ability acknowledges past research that shows a clear gender differentiation for mechanical ability. Schools and society should address such biases for certain abilities.
Scientific ability, a hybrid involving conceptualizing, memory, and perhaps interest in the area, requires early identification because of the hierarchical way the ability is nurtured and developed within our educational system. Developing scientific ability after little exposure is more difficult for people in their late teens and early adulthood than at a chronologically earlier age. The critical dimension is a person's exposure and identification with the unique subset of skills as being doable for him or her. Self-efficacy beliefs or feelings of adequacy, both of which can be part of the ability construct, need examination.
Readers will find the above abilities in the literature but with different names. In a summary of 25 years of research, Prediger (1992) reported the same major skills, except that he identified literary rather than the musical/dramatic ability listed above. Lowman (1991) included in his literature review 11 of the above abilities as having career relevance. He did not list four of the abilities as major abilities--scientific, reading, social, and persuasive. Instead he cited intelligence as more predictive for science occupations. He wrote, "Interpersonal skills or social intelligence appears not to be a unidimensional construct" (p. 109). He set forth a taxonomy of social demands, however, that clearly differentiates interpersonal from helping skills, which require the ability to understand the behavior and feelings of others. Lowman expressed that personal factors are most important in predicting sales performance. So science, social, and persuasive domains were recognized but were not attributed as primary abilities. Reading and language were cited among the small number of factors found in the verbal factor.
Common existing tests measure six to eight of the abilities listed in the first column, above. This narrow band of abilities emerged from the multi-aptitude measures, mostly developed in the late 1940's. Job analysts, on the other hand, identify many of the aptitudes listed in the second column as necessary abilities for some jobs. Unfortunately, young people are evaluated on these abilities and educators seldom identify them for self exploration.
It should be mentioned that knowledge of an individual's ability profile may be of moot value. Hunter (1986), after reviewing hundreds of studies, wrote "...cognitive ability predicts job performance in all jobs...including the so-called 'manual' jobs as well as 'mental' jobs" (p. 340). He continued, "Cognitive ability predicts job performance in large part because it predicts learning and job mastery. Ability is highly correlated with job knowledge and job knowledge is highly correlated with job performance" (p. 354).
If They Are Important, Why Haven't Tests of These Abilities Been Available?
The regression model, which minimizes the number of tests used in predicting success, has dominated the field of ability measurement. Goldman (1972), among others, pointed out that multiple aptitude batteries have limited differential predictive value and they do not offer much more than an intelligence or academic ability test. He felt that multiple aptitude tests have little to offer in counseling clients in their decision making and career planning. He wrote, "The main contribution of tests in counseling is not making predictions but facilitating the clarification of self concept" (p. 219). The National Commission on Testing and Public Policy (1990) also called for the transformation of testing in America from a gatekeeper to that of a facilitator. The Commission stated testing programs must change from an over reliance on objective tests to alternative forms of assessment that help people become aware of and develop their talents. With most state plans for career development calling for students to record data about their abilities, a longer list of abilities is relevant for life planning.
What Is Self-Report Methodology And How Does Its Validity Compare With The Traditional Approach?
Three different self-report assessment formats have been used. One is simply a listing of abilities with definitions and directions to indicate those areas you feel are your best or strongest. A second approach is to apply a Likert scale to a group of designated abilities. For example, in comparison to others of the same age, my art ability is excellent, above average, average, below average, or poor. Another approach is, for each ability, to provide different examples of the ability's applications on which individuals rate their performance level from high to low, and subsequently these are summed to obtain a total score. Whereas most multiple ability testing situations need several hours, the time required for the above formats ranges from 10 to 45 minutes.
Self-report assessment is cheaper and less time intrusive on a school's schedule. How do the approaches compare regarding validity? Ghiselli's (1973) summary of the average validity coefficients of different kinds of aptitude tests used to predict proficiency in the eight major occupational categories of the General Occupational Classification System shows that the coefficients are typically in the .20's and rarely go above the .30's. The average validity coefficient for prediction of proficiency on the job was .22. In a review of 55 self-evaluation of ability scales, Mabe and West (1982) reported a range of correlation coefficients from -.026 to .80, with a mean coefficient of .29 (depending on the meta-analytic method used).
More recently, Westbrook, Buck, Sanford, and Wynne (1994) demonstrated that it is possible to get acceptable reliability and validity coefficients for self ratings which approach the size of the validity coefficients reported for objective measures of ability. Their comparative measure was the Differential Aptitude Test (DAT). In another study based on the common criterion of self-reported abilities of employees, Harrington and Schafer (in press) compared the abilities required for jobs from Guide for Occupational Exploration (GOE) job analysis data with the General Aptitude Test Battery's (GATB) Occupational Aptitude Patterns (OAP). The GOE and OAP average percentages were compared in order to evaluate which was more consistent with workers' self expressions of their abilities. Across the 51 occupations studied, 49 of the GOE averages were larger versus one in which the OAP average was larger. It was concluded that the GOE analysis data are more congruent with worker-identified job abilities than the GATB analyses.
Conclusion
Current use of self-assessment methodology taps more ability areas than existing ability or aptitude tests cover. Alternative testing approaches have been called for which enhance self discovery and awareness. Some recent self-report studies show at least comparable validity with more traditional approaches. Some researchers are advocating the self-assessment methodology which can substantially cut loss of instructional time and cost, evaluate hard-to-assess constructs, and deliver information most people feel is useful for self knowledge and career planning. Philosophically, the process of self evaluation fits the belief that individuals are in the best position to assess since they have access to a large data base on their own successes and failures in their abilities. Most misgivings about the methodology seem to center around beliefs that individuals have a tendency to be lenient and are not objective enough in their self analysis to provide accurate self reports.
References
Thomas F. Harrington is Program Director of the Counseling Psychology Program at Northeastern University, Boston, MA.
This digest was created by ERIC, the Educational Resources Information Center. For more information about ERIC, contact ACCESS ERIC 1-800-LET-ERIC.
ERIC Digests are in the public domain and may be freely reproduced and disseminated. This publication was funded by the U.S. Department of Education, Office of Educational Research and Improvement, Contract No. RR93002004. Opinions expressed in this report do not necessarily reflect the positions of the U.S. Department of Education, OERI, or ERIC/CASS.