By Bruce L. Katcher, Ph.D. President, Discovery Surveys, Inc.

Only half of all employees say they understand the measures used to assess their job performance.

Those of you who regularly read this column know that my sister has had a succession of terrible jobs. Here's the latest tale of woe in her long tumultuous employment history.

She was recently placed on 60-day probation and may lose her job as a collections specialist for a medical equipment rental company because she has a low, what they call, "collection ratio." They monitor the percentage of the outstanding balances assigned to her that she collects each month. She is actually doing a very good job of collecting, but her ratio doesn't reflect it. There is a problem in how it is computed. Any account with a balance of $5,000 or more is assigned to her supervisor who is doing a poor job of collecting this money. But, the company has no way to separate out my sister's collection ratio from her supervisor's ratio. The result is that my sister's ratio is misleadingly low.

When she brought this to the attention of management she was told, "There is nothing we can do. It would take 26 special reports to untangle it all and we're just not going to do it. We're all a team here. Aren't you a team player?"

Determined to keep her job, she offered to help her supervisor collect the larger balances. The supervisor, who by the way is not on 60-day probation, declined her offer. Does this make sense to you? It doesn't to me.

THE PROBLEM:

If organizations are going to judge employees based on their job performance, they must have a legitimate way to accurately measure performance. The truth is, however, that organizations have a difficult time correctly measuring job performance. Here are a few examples of common reasons why measures of job performance are faulty.

• Opportunity Bias - A salesperson complains that the way his sales success is measured should take into account the fact that his sales territory is much smaller than the territories of other salesmen in the company and he, therefore, has less of an opportunity to be successful.

• Halo - A marketing manager complains that her performance reviews are always low because her supervisor just doesn't like her, not because of the quality of her job performance. She rates her based on a general overall impression.

• Criterion Contamination - A claims adjuster for an auto insurance company argues that the large volume of expensive claims he processed for the company was not his fault. It was due to events out of his control (i.e., the snowy weather which led to an unusually large number of accidents his policyholders had during the winter).

• Leniency - A teller in a local bank complains that everyone in her branch received lower bonuses compared to the branches in the surrounding towns because their supervisor had higher expectations than the other supervisors. (Note: the term leniency refers to both artificially inflated and deflated ratings.)

• Deficiency - The performance of sales personnel at a consumer electronics store is calculated from the ratings they receive from customer satisfaction surveys. The problem, however, is that after a customer purchase, the surveys are mailed to their homes and not all of the customers complete and return it. The sales personnel feels that those who are satisfied just don't bother to fill it out.

• Irrelevancy - The performance of junior consultants is typically based on the number of billable hours they assign to clients. However, they have no sales responsibilities and are totally dependent on others to bring in the work that they then conduct. Their billable hours are low because others aren't doing a good job of selling, not because they aren't trying to do as much work as possible.

• Group Rather than Individual Measures - Many organizations make judgments about individual performance based on the group's performance. For example, you might be the star employee in your department, but if the department is not meeting its goals, your individual performance rating will suffer.

• The Bell Curve Backfire - Some organizations use a forced distribution rating approach. For example, if there are 10 people in the department, the supervisor is told that only 2 can receive a rating of "Exceeds Expectations" and that 5 or 6 must receive the middle rating of "Meets Expectations." This is supposed to solve the problem of leniency. However, it may be that there really are more than 2 employees who are exceeding expectations.

• Dishonesty - Many organizations now use a 360-feedback approach to measuring performance. In addition to supervisors, an employee's direct reports and peers are also asked to rate their performance. The problem is that, for their own selfish reasons, these raters may be less than honest. For example, direct reports might fear retribution from their supervisors and peers might worry about damaging fragile collegial relationships or receiving low ratings from that employee when the tables are turned and it is their turn to be rated.

Getting back to my sister, the lone measure that is being used to assess her job performance is contaminated, deficient, and irrelevant. (It is also insulting.) There are probably ulterior motives at play here as well.

WHAT TO DO

1. Try to Develop Pure Quantitative Measures of Job Performance

   Be creative. Just because they haven't been used before doesn't mean they can't be introduced now. For many jobs, useful metrics can be created for a variety of performance indices such as sales, speed, errors, cost control, efficiency, customer complaints, external customer satisfaction, internal customer satisfaction, quality, and quantity of production. Make sure these measures are appropriate for the job and avoid contain no obvious sources of contamination, bias, deficiency or irrelevancy.

2. Continually Train Managers

   Managers need to be trained regularly on the proper use of performance rating instruments. They need to be continuously educated about sources of potential errors in their ratings such as halo, leniency, and bias.

3. Be Cautious If Using 360 Feedback

   Receiving feedback from peers, subordinates and others can be a valuable developmental exercise for employees. However, basing personnel decisions such as pay, promotions, probation, and termination on such information is problematic. The motivation of peers and subordinates is not the same as that of an immediate supervisor who is accountable to the organization for providing an accurate performance rating.

4. Use Multiple Measures

   It is rare that one type of performance measure such as an individual's sales figures or his or her supervisory rating provides an accurate assessment of an individual's total performance. A variety of different types of measures should be

5. Involve Employees in Developing the Measures

   Employees should know ahead of time how their performance will be measured. Ideally they will be given a say in how their performance is measured. They can do this by helping to set their own performance goals.

CONCLUSION

You're not being fair to your employees if you are using irrelevant, contaminated or biased measures of job performance. If necessary admit the failure in your method and develop a different approach for making intelligent decisions about pay or whether or not employees should be placed on probation.

Oh, and by the way, if you know of a good job for my sister in Northern New Jersey, please let me know. She is a very detail oriented team player with experience in accounts receivable, collections, general ledger, and accounting. She is also an excellent cook with catering experience.