Author here, great to see all the conversation & thoughts you all have shared so far!
One thing I've seen a lot of people saying: hiring juniors isn't worth it because they'll just leave for more money in a couple years.
I got hired with 0 experience for my first job and stayed for 3.5 years (I left when I decided to start my own company). I was otherwise never tempted to leave because I got all the support and growth opportunities I could have hoped for, and I always felt fairly compensated.
So based on my own experience, it's possible for a company to treat people well enough that they won't just leave. I do think software companies tend to be bad at this specific thing though - I'd imagine many have had an experience unlike my own.
From my experience, junior's tend to churn out after around three years. There are often factors that go beyond whether they felt they were treated well. Looking at it holistically, younger people tend to want to move around more. It is no surprise to me that the average tenure of a junior employee is about the same length as a degree - separating life stages in to 1-3 year periods is a mindset embedded into young people throughout their education.
I would wager that perception plays heavily into this too. It can be difficult to shake a perception of "being a junior", especially when the path to seniority is unclear or poorly defined. Plus, the "two years and disappear" ethos of job hopping for quicker compensation capitalises on companies' conservative promotion criteria (and more liberal hiring criteria). Loyalty is rarely rewarded in these cases.
If you keep asking questions eventually you hit something they don't actually know. How long it takes to get there + what they say when they get there is what makes this interesting imo!
Great idea re: giving hard problems. Same motivation behind why we ask people about past projects and keep diving deeper and deeper. The point is to figure out if they're curious & capable of engaging on a deeper level, vs. just following a tutorial they found somewhere.
You're totally right! And we do not account for it at all. We don't give any insights into product decisions, just engineering output. If engineering output is high but business results are bad, that indicates there's a problem somewhere else in the business. But it's still good to know that engineering output is high!
> Let me just ignore my natural distain to the whole thing (as a engineer and a manager)
I totally get this - that's how I felt initially, but I was shocked to find that the vast majority of orgs are using bad metrics like LOC or commit counts anyway. Our belief is that replacing those with something much more accurate can help the entire industry.
I worked as a manager in a big tech company that used metrics such as amount of PRs done, amount of PR reviews and etc. At least with those when there were questions based on those metrics - I was able to contextualize and explain to upper management why for a particular eng they show a dip even when their performance is stellar.
How am I going to do that with your blackbox metrics when this need arises?
Also I don't have a Google account, so I can't even get pass your frontpage that has no info?
Different companies will have different outputs just by the nature of their stage & situation, but the numbers are still relatively comparable (e.g. across different teams).
> How do you measure the impact of refactoing?
The metric def gives credit for refactoring
> What about regressions or design mistakes that surface themselves after months or even years?
Not captured (part of why it's only an important part of the story, not the whole story :))
LOL - I shudder at the idea of a manager making HR decisions based solely on this one metric!
To be clear we're not claiming this is 1 number to holistically evaluate an entire engineer. Rather we're giving a much more accurate picture of output, which most orgs are already measuring (with terrible accuracy). It should be an important part of the picture but certainly not the whole story!
And fwiw I think the scores are pretty transparent - in the platform, you can drill down into any number and see the actual PRs and their output measurements. Of course the underlying model is more complex but unfortunately simpler models are not sufficient to capture the way engineering output works.
One thing I've seen a lot of people saying: hiring juniors isn't worth it because they'll just leave for more money in a couple years.
I got hired with 0 experience for my first job and stayed for 3.5 years (I left when I decided to start my own company). I was otherwise never tempted to leave because I got all the support and growth opportunities I could have hoped for, and I always felt fairly compensated.
So based on my own experience, it's possible for a company to treat people well enough that they won't just leave. I do think software companies tend to be bad at this specific thing though - I'd imagine many have had an experience unlike my own.