THE ONGOING SAGA OF MY SISTER'S
By Bruce L. Katcher, Ph.D. President, Discovery Surveys, Inc.
Only half of all employees say
they understand the measures used to assess their job performance.
Those of you who regularly read this column know that my sister
has had a succession of terrible jobs. Here's the latest tale of woe in her
long tumultuous employment history.
She was recently placed on 60-day probation and may lose her job
as a collections specialist for a medical equipment rental company because she
has a low, what they call, "collection ratio." They monitor the percentage of
the outstanding balances assigned to her that she collects each month. She is
actually doing a very good job of collecting, but her ratio doesn't reflect it.
There is a problem in how it is computed. Any account with a balance of $5,000
or more is assigned to her supervisor who is doing a poor job of collecting
this money. But, the company has no way to separate out my sister's collection
ratio from her supervisor's ratio. The result is that my sister's ratio is
When she brought this to the attention of management she was told,
"There is nothing we can do. It would take 26 special reports to untangle it
all and we're just not going to do it. We're all a team here. Aren't you a team
Determined to keep her job, she offered to help her supervisor
collect the larger balances. The supervisor, who by the way is not on 60-day
probation, declined her offer. Does this make sense to you? It doesn't to me.
If organizations are going to judge employees based on their job
performance, they must have a legitimate way to accurately measure performance.
The truth is, however, that organizations have a difficult time correctly
measuring job performance. Here are a few examples of common reasons why
measures of job performance are faulty.
- Opportunity Bias
A salesperson complains that the way his sales
success is measured should take into account the fact that his sales territory
is much smaller than the territories of other salesmen in the company and he,
therefore, has less of an opportunity to be successful.
A marketing manager complains that her
performance reviews are always low because her supervisor just doesn't like
her, not because of the quality of her job performance. She rates her based on
a general overall impression.
- Criterion Contamination
A claims adjuster for an auto insurance company
argues that the large volume of expensive claims he processed for the company
was not his fault. It was due to events out of his control (i.e., the snowy
weather which led to an unusually large number of accidents his policyholders
had during the winter).
A teller in a local bank complains that everyone
in her branch received lower bonuses compared to the branches in the
surrounding towns because their supervisor had higher expectations than the
other supervisors. (Note: the term leniency refers to both artificially
inflated and deflated ratings.)
The performance of sales personnel at a consumer
electronics store is calculated from the ratings they receive from customer
satisfaction surveys. The problem, however, is that after a customer purchase,
the surveys are mailed to their homes and not all of the customers complete and
return it. The sales personnel feels that those who are satisfied just don't
bother to fill it out.
The performance of junior consultants is typically
based on the number of billable hours they assign to clients. However, they
have no sales responsibilities and are totally dependent on others to bring in
the work that they then conduct. Their billable hours are low because others
aren't doing a good job of selling, not because they aren't trying to do as
much work as possible.
- Group Rather than Individual Measure-
Many organizations make judgments about individual
performance based on the group's performance. For example, you might be the
star employee in your department, but if the department is not meeting its
goals, your individual performance rating will suffer.
- The Bell Curve Backfire
Some organizations use a forced distribution
rating approach. For example, if there are 10 people in the department, the
supervisor is told that only 2 can receive a rating of "Exceeds Expectations"
and that 5 or 6 must receive the middle rating of "Meets Expectations." This is
supposed to solve the problem of leniency. However, it may be that there really
are more than 2 employees who are exceeding expectations.
Many organizations now use a 360-feedback approach
to measuring performance. In addition to supervisors, an employee's direct
reports and peers are also asked to rate their performance. The problem is
that, for their own selfish reasons, these raters may be less than honest. For
example, direct reports might fear retribution from their supervisors and peers
might worry about damaging fragile collegial relationships or receiving low
ratings from that employee when the tables are turned and it is their turn to
Getting back to my sister, the lone measure that is being used to
assess her job performance is contaminated, deficient, and irrelevant. (It is
also insulting.) There are probably ulterior motives at play here as well.
WHAT TO DO
- Try to Develop Pure Quantitative Measures of
Be creative. Just because they haven't been used
before doesn't mean they can't be introduced now. For many jobs, useful metrics
can be created for a variety of performance indices such as sales, speed,
errors, cost control, efficiency, customer complaints, external customer
satisfaction, internal customer satisfaction, quality, and quantity of
production. Make sure these measures are appropriate for the job and avoid
contain no obvious sources of contamination, bias, deficiency or irrelevancy.
- Continually Train Managers
Managers need to be trained regularly on the
proper use of performance rating instruments. They need to be continuously
educated about sources of potential errors in their ratings such as halo,
leniency, and bias.
- Be Cautious If Using 360 Feedback
Receiving feedback from peers, subordinates and
others can be a valuable developmental exercise for employees. However, basing
personnel decisions such as pay, promotions, probation, and termination on such
information is problematic. The motivation of peers and subordinates is not the
same as that of an immediate supervisor who is accountable to the organization
for providing an accurate performance rating.
- Use Multiple Measures
It is rare that one type of performance measure
such as an individual's sales figures or his or her supervisory rating provides
an accurate assessment of an individual's total performance. A variety of
different types of measures should be
- Involve Employees in Developing the Measures
Employees should know ahead of time how their
performance will be measured. Ideally they will be given a say in how their
performance is measured. They can do this by helping to set their own
You're not being fair to your employees if you are using
irrelevant, contaminated or biased measures of job performance. If necessary
admit the failure in your method and develop a different approach for making
intelligent decisions about pay or whether or not employees should be placed on
Oh, and by the way, if you know of a good job for my sister in
Northern New Jersey, please let me know. She is a very detail oriented team
player with experience in accounts receivable, collections, general ledger, and
accounting. She is also an excellent cook with catering experience.
I am very much
interested in your views on this topic.
Please reply with your comments and
suggestions to .
TOP OF PAGE
All material is © copyright , Discovery Surveys, Inc.,