Performance Appraisal - Methods of Collecting Performance Appraisal Data - Judgmental Evaluation

Judgmental Evaluation

Judgmental evaluation appears to be a collection of methods, and as such, could be considered a methodology. A common approach to obtaining PAs is by means of raters. Because the raters are human, some error will always be present in the data. The most common types of error are leniency errors, central tendency errors, and errors resulting from the halo effect. These errors arise predominantly from social cognition and the theory in that how we judge and evaluate other individuals in various contexts is associated with how we “acquire, process, and categorize information”.

An essential piece of this method is rater training. Rater training is the “process of educating raters to make more accurate assessments of performance, typically achieved by reducing the frequency of halo, leniency, and central-tendency errors”. Rater training also helps the raters “develop a common frame of reference for evaluation” of individual performance. Many researchers and survey respondents support the ambition of effectual rater training. However, it is noted that such training is expensive, time consuming, and only truly functional for behavioral assessments.

Another piece to keep in mind is the effects of rater motivation on judgmental evaluations. It is not uncommon for rating inflation to occur due to rater motivation (i.e. “organizationally induced pressures that compel raters to evaluate ratees positively”). Typically, raters are motivated to give higher ratings because of the lack of organizational sanction concerning accurate/inaccurate appraisals, the rater's desire to guarantee promotions, salary increases, etc., the rater's inclination to avoid negative reactions from subordinates, and the observation that higher ratings of the ratees reflect favorably upon the rater.

The main methods used in judgmental performance appraisal are:

• Graphic Rating Scale: graphic rating scales (see scale (social sciences)) are the most commonly used system in PA. On several different factors, subordinates are judged on 'how much' of that factor or trait they possess. Typically, the raters use a 5- or 7-point scale; however, there are as many as 20-point scales.

• Employee-Comparison Methods: rather than subordinates being judged against pre-established criteria, they are compared with one another. This method eliminates central tendency and leniency errors but still allows for halo effect errors to occur. The rank-order method has raters ranking subordinates from “best” to “worst”, but how truly good or bad one is on a performance dimension would be unknown. The paired-comparison method requires the rater to select the two "best" subordinates out of a group on each dimension then rank individuals according to the number of times each subordinate was selected as one of the "best". The forced-distribution method is good for large groups of ratees. The raters evaluate each subordinate on one or more dimensions and then place (or ”force-fit”, if you will) each subordinate in a 5 to 7 category normal distribution. The method of top-grading can be applied to the forced distribution method. This method identifies the 10% lowest performing subordinates, as according to the forced distribution, and dismisses them leaving the 90% higher performing subordinates.

• Behavioral Checklists and Scales: behaviors are more definite than traits. The critical incidents method (or critical incident technique) concerns “specific behaviors indicative of good or bad job performance”. Supervisors record behaviors of what they judge to be job performance relevant, and they keep a running tally of good and bad behaviors. A discussion on performance may then follow. The behaviorally anchored rating scales (BARS) combine the critical incidents method with rating scale methods by rating performance on a scale but with the scale points being anchored by behavioral incidents. Note that BARS are job specific.

Peer and Self Assessments

While most judgmental PA research is evaluated by a superior (e.g. supervisor, manager), peer assessments are evaluated by one’s colleagues. With self-assessments, individuals evaluate themselves.

• Peer Assessments: members of a group evaluate and appraise the performance of their fellow group members. There are three common methods of peer assessments. Peer nomination involves each group member nominating who he/she believes to be the “best” on a certain dimension of performance. Peer ratings has each group member rate each other on a set of performance dimensions. Peer ranking requires each group member rank all fellow members from “best” to “worst” on one or more dimensions of performance.

• Self-Assessments: for self-assessments, individuals assess and evaluate their own behavior and job performance. It is common for a graphic rating scale to be used for self-assessments. Positive leniency tends to be a problem with self-assessments.

• 360-Degree Feedback: 360-degree feedback is multiple evaluations of employees which often include assessments from superior(s), peers, and one’s self.