Use Git data to optimize your developers’ annual reviews

The end of the year is looming and with it one of your most important tasks as a manager. Summarizing the performance of 10, 20 or 50 developers over the past 12 months, offering personalized advice and having the facts to back it up — is no small task.

We believe that the only unbiased, accurate and insightful way to understand how your developers are working, progressing and — last but definitely not least — how they’re feeling, is with data. Data can provide more objective insights into employee activity than could ever be gathered by a human.

Consider this: Over two-thirds of employees say they would put more effort into their work if they felt more appreciated, and 90% want a manager who’s fair to all employees.

Let’s be honest. It’s hard to judge all of your employees fairly if you’re (1) unable to work physically side-by-side with them, meaning you’ll inevitably have more contact with the some over others (e.g., those you’re more friendly with); and (2) you’re relying on manual trackers to keep on top of everyone’s work, which can get lost and take a lot of effort to process and analyze; (3) you expect engineers to self-report their progress, which is far from objective.

It’s also unlikely, especially with the quieter ones, that on top of all that you’ll have identified areas for them to expand their talents by upskilling or reskilling. But it’s that kind of personal attention that will make employees feel appreciated and able to progress professionally with you. Absent that, they’re likely to take the next best job opportunity that shows up.

So here’s a run down of why you need data to set up a fair annual review process; if not this year, then you can kick-start it for 2021.

1. Use data to set next year’s goals

The best way to track your developers’ progress automatically is by using Git Analytics tools, which track the performance of individuals by aggregating historical Git data and then feeding that information back to managers in minute detail.

This data will clearly show you if one of your engineers is over capacity or underworked and the types of projects they excel in. If you’re assessing an engineering manager and the team members they’re responsible for have been taking longer to push their code to the shared repository, causing a backlog of tasks, it may mean that they’re not delegating tasks properly. An appropriate goal here would be to track and divide their team’s responsibilities more efficiently, which can be tracked using the same metrics, or cross-training members of other teams to assist with their tasks.

Another example is that of an engineer who is dipping their toe into multiple projects. Indicators of where they’ve performed best include churn (we’ll get to that later), coworkers repeatedly asking that same employee to assist them in new tasks and of course positive feedback for senior staff, which can easily be integrated into Git analytics tools. These are clear signs that next year, your engineer could be maximizing their talents in these alternative areas, and you could diversify their tasks accordingly.

Once you know what targets to set, you can use analytics tools to create automatic targets for each engineer. That means that after you’ve set it up, it will be updated regularly on the engineer’s progress using indicators directly from the code repository. It won’t need time-consuming input from either you or your employee, allowing you both to focus on more important tasks. As a manager you’ll receive full reports once the deadline of the task is reached and get notified whenever metrics start dropping or the goal has been met.

This is important — you’ll be able to keep on top of those goals yourself, without having to delegate that responsibility or depend on self-reporting by the engineer. It will keep employee monitoring honest and transparent.

2. Three Git metrics can help you understand true performance quality

The easiest way for managers to “conclude” how an engineer has performed is by looking at superficial output: the number of completed pull requests submitted per week, the number of commits per day, etc. Especially for nontechnical managers, this is a grave but common error. When something is done, it doesn’t mean it’s been done well or that it is even productive or usable.

Instead, look at these data points to determine the actual quality of your engineer’s work:

1. Churn is your number-one red flag, telling you how many times someone has modified their code in the first 21 days after it has been checked in. The more churn, the less of an engineer’s code is actually productive, with good longevity. Churn is a natural and healthy part of the software development process, but we’ve identified that any churn level above the normal 15%-30% indicates that an engineer is struggling with assignments.

1. A high amount of follow-on commits in pull requests is also an indicator of low quality of engineers’ work. As is low impact. Impact is a complex metric telling you the amplitude of code changes implemented by one engineer (and therefore the cognitive load they carried when doing so). When this falls, it means their code is having less real impact on the final codebase.

2. High-risk commits are also worth mentioning. They’re not necessarily negative, but they increase the chances of the code causing problems, for example because deep — rather than trivial — edits are being made to it. A high amount of high-risk commits indicates the fact that a developer is not adhering to best practices.

3. Compare engineers to their past performance, not against others

It may sound obvious, but it’s still very hard for many managers to fully understand that all employees work at different paces and levels. One might be pumping out code like crazy, but they insert far more bugs than the slow and steady engineer who submits half the amount of code per week, but requires almost no amends. It’s simply easier for managers to look at the same metrics across teams, but that would be an unfair way to assess how everyone has performed this year.

When you’re looking at the aforementioned work quality metrics, they’re not black and white. It’s not right to compare the churn for a junior engineer who’s working their first job in your company, to that of another junior who has already worked in similar fields for a year. If your new employee has drastically improved their efficiency over the past 12 months (even if it’s still lower than average), it actually shows they are a fast learner and have a lot of promise for the future. If you simply compare their output to that of their coworkers, you’ll probably write them off and lose out.

Git analytics tools are highly visual. Unlike with purely qualitative data, you won’t need to do much mental gymnastics to understand if someone is getting better or worse with time. Trend lines can show you how an engineer has performed for each individual metric, compared to the last sprint, quarter or year.

When you’re actively acknowledging that everyone has different work styles, you’ll also reduce toxic competitiveness between colleagues and promote a healthier, personal motivation to continue improving performance.

4. Subjective judgment versus fair feedback and reward

Over 95% of managers probably make decisions with a large dolloping of intuition, because they don’t have enough data to back up their judgment. A survey from back in 2002 revealed that 45% of corporate executives actually relied more on instinct than on facts when running their business.

It works like this. If a manager engages frequently with an engineer who appears more focused — they’re chipping in in every meeting, they have closed more tasks — the manager may assume that they’re producing more value for the company. While an introvert who is not as open about their work and is less communicative might be producing far better quality work (which means their superiors are less likely to hear about them — “no news is good news”). Yet the manager may assume that they’re not excelling.

New tools mean this is now changing, because data can be compared to the same engineer’s previous period. Furthermore, with more facts on hand, managers will be able to remove the subjective sentiments out of their yearly reviews and simply state the facts.

This is key when deciding how to divvy up your end-of-year bonuses. The bigger slices should go to those who have gone above and beyond to not only further your business goals but to help others. Consider checking how many different projects an engineer has been contributing to (on top of their regular tasks) to help others achieve their goals.

5. Hard worker versus burnout risk

Finally, we need to talk about mental health. Annual reviews aren’t just about performance; they’re a chance for you to hash out serious issues your engineers may be having as a result of their jobs.

A clear indicator here is spikes. When someone’s work routine is unpredictable — for example, their productivity is spiking far above their average for two or three days a week, then they’re practically inactive the rest of the week — it’s likely they’re burdening themselves with too much at a short deadline. It could also mean their work life is at complete odds with their personal life, forcing them to work ridiculous hours.

Use this indicator to look at the distribution of work within teams and within someone’s personal work week. Crucially, use the annual reviews to collect honest feedback from your engineers on how this year and the move to WFH has affected them, and how the company can make them feel the most comfortable at work. Remote work has made our interactions more transactional: assign and deliver, whittling away at company culture. This is especially true of startups compared to big companies with an established culture.

Hopefully, 2021 will see you maturing into better habits. At Waydev, we spend at least one hour a week catching up with employees for nonwork meetings. We also “shut down” at 6 p.m. to give people time to have a personal life. As an engineer, you can technically work all the time. But as a manager, even from afar you can impose some discipline with working habits, finding a middle ground between order and giving employees the flexibility to work at their own pace.

As a senior employee, remember that mental health always comes back to the health of the company. Pushing your engineers too far may get some short-term results, but it will quickly affect cadence. Without cadence, it’s extremely hard to plan and manage teams.

You don’t have to have the equivalent of a yearly review every few weeks to know if your engineers are doing well. By setting up a data-driven project management system, you can take a step back and only interrupt when those red flags show up. Instead, take the time to assess how your engineers might want to improve their career within your company in the future. Use the review to ask them what they aspire to and how you can both be moving forward in unison.

https://techcrunch.com/2020/12/03/vcs-who-want-better-outcomes-should-use-data-to-reduce-founder-team-risk/