If your company does performance reviews and is relatively large, there is a good chance that there is a guiding bell curve used to validate the distribution of different performance ratings. The idea is that for large-enough groups, the distribution of performance should typically fit a left-skewed bell curve. There is some debate on what “large-enough” means. Some set the threshold at 30 people, others at 100.

A good guiding curve is like a smoke detector. It tells you there is smoke, but you can’t be sure there is a fire without inspection. It leads to questions like “Are managers being consistent in their performance reviews?” and “Does this right-skewed distribution on Team A indicate a manager who is too harsh or a low-performing team?“ In my experience, these guiding curves are largely despised by managers, especially when they feel forced to downgrade a rating to fit the curve.

In the 2012 paper “The best and the rest: Revisiting the norm of normality of individual performance,” researchers looked into the performance of more than 600 thousand people in four large professional groups: athletes, academics, politicians, and entertainers.

"Our results clearly support the superiority of the Paretian distribution compared to the Gaussian distribution to model individual performance."

The Gaussian distribution is the bell curve many, if not most, companies use.

The Paretian distribution is also known as the 80/20 rule.

The authors of the paper find that “superstar” performers have a disproportionally large impact (below in green) and that the rest of the population forms a long tail (below in yellow).

While “superstar” and “the rest” seem polarizing and elitist, the authors address the importance of supporting roles.

"[S]uperstars likely cannot emerge in a vacuum. Top researchers can devote the necessary time to their work because there are others who take on some of their teaching and administrative duties. Hollywood stars emerge in part because of supporting casts on screen as well as off screen (e.g., agents, managers, publicists)."

If you are excited about ditching the bell curve in your company’s performance cycle, consider this significant challenge: Paretian distributions give you two groups (e.g., superstars and “the rest”), but a typical guiding curve is based on a three to five-point rating system, like:

Exceeds expectations
Meets expectations
Below expectations

One solution is to group “Meets expectations” and “Below expectations” as “the rest” - Good luck convincing HR to do that!

Now what?

These findings illustrate how artificial it can be to set distribution targets that divide “Meets expectations” and “Below expectations” ratings.

If you've been working to change the guiding curve in your review cycles, you can use this paper to persuade your company to move toward a more straightforward inspection method. The actual ratings can continue to follow the three to five-point system. But, the guiding distribution would be a Paretian distribution with only two groups - and, thankfully, you don't have to use the label superstar.

If you want to go one step further, an alternative is the Adobe Check-in Model. They solved this problem in 2012 by not having any formal ratings or rankings.

Thank you for reading!
Thiago

P.S. I’ve 2x-ed my posting frequency this year. Going from one post per year since 2020 to the incredible figure of two posts in 2022 alone! If you find this content useful, please consider subscribing to the premium edition. It costs you the equivalent of buying me a beer per month (yes, a nice beer) and will support the planned 12x increase in posting frequency per year - plus the podcast and other fun things. Thanks again!

Subscribe now

Now what, Tech Lead?

Paretian distributions model individual…