Company blog of DX, a developer productivity analytics firm now part of Atlassian, covering engineering metrics, developer experience and platform teams.
HN commenters debate its Core 4 and SPACE metrics, with some objecting to measuring individual developers, and others discussing AI's effect on productivity.
Let's also not forget the multiple other studies that found significant boosts to productivity using rigorous methods like RCTs.
However, because these threads always go the same way whenever I post this, I'll link to a previous thread in hopes of preempting the same comments and advancing the discussion! https://news.ycombinator.com/item?id=46559254
Also, DX (whose CTO was giving the presentation) actually collects telemetry-based metrics (PR's etc.) as well: https://getdx.com/uploads/ai-measurement-framework.pdf
It's not clear from TFA if these savings are self-reported or from DX metrics.
But also not directly to developer's experience.
Try the SPACE framework [1] from the same people, Nicole Forsgren et al.
It is more of a survey than measurable objective metrics, and the problem is with the implementation, like any other survey- build good questions, operating the survey analysis and results etc.
There some comercial tools available, for example getdx [2]
[1] https://queue.acm.org/detail.cfm?id=3454124
[2] https://getdx.com/
To the author’s point about there not being enough evidence to support pro-AI claims — Additionally, there are studies and articles coming out showing those people are wrong and AI only makes people barely more efficient. Not the 10x tech CEOs are claiming while slashing thousands of jobs
https://getdx.com/blog/ai-productivity-gains-are-10-percent-not-10x/
You can't do all this alone; you need allies. So use your communication skills to persuade a suitable sponsor with the power to assign the necessary resources. Google and DX regularly release reports about these things; e.g., https://getdx.com/blog/documentation-culture-engineering/
You could use them to raise awareness in your organization.
> Core 4 (and similar) rub me the wrong way – measuring individual developers as the atomic unit IMO is always meaningless
And From: https://getdx.com/research/measuring-developer-productivity-with-the-dx-core-4/
> Diffs per engineer*
>
> * Not at individual level
Are you guys agreeing or disagreeing with each other?
It's a combination of different activity metrics (commits, PRs, comments, etc) with AI thrown in to figure out substance/complexity of the work. I originally thought the summaries were simple metrics without factoring quality. There are products that provide this for github and other tools. E.g. https://getdx.com/