The State of Python 2025

In "real" polling, systematic bias is somewhat corrected for by a series of processes broadly known as "weighting." The general idea is that you look at high-quality demographic data that you have reason to believe is accurate for your intended sample space (e.g. because it came from a census, hospital records, or other reliable sources of aggregate information), compare that data against the data from your own survey, and adjust the weight given to each response until your surveyed demographics roughly agree with reality. There are numerous problems with this, and it is quite far from a silver bullet, but it is likely better than doing nothing. Weighting has error bars of its own, and for the data nerds, you probably should break those out separately in the crosstabs or raw results, but the error bars on the headline number (i.e. "candidate X leads with Y% of the vote" or whatnot) usually will account for all potential sources of bias that have been considered (or at least, all the bias that is reasonably possible to quantify, anyway).

One of the problems with weighting is that it only works if you have good demographic data to begin with. That's a somewhat believable assumption when your sample space is "the population of country X," but not when it's "everyone who writes code in Python." So it's really difficult to apply weighting to surveys like this, and my default assumption is that it has not been done.

Looking at the the venues where this survey was promoted according to https://lp.jetbrains.com/python-developers-survey-2024/#m... , my guess is a lot of people who use python for tooling were missed (including me.)

Everything else aside, I think you could find much better evidence to question AI agents. For instance the recent METR randomized controlled trial (RCT) where people thought AI made them 30% faster, when in fact they were 20% slower than the control group.

Is driving vastly simpler than a grand-master level at chess, evidenced by the fact that many more people can drive? For humans, certainly. But my mid-range phone from a decade ago is much stronger still than a grandmaster. The point here is that human abilities don't correlate to computer abilities. You can't use computer performance in one human task to guess the performance the computer will have in other tasks.

What you have here is good evidence that you should *ignore* hype. Merely reacting in the opposite direction means you are still being influenced by hype. What we should do is judge new things on their own merits, regardless of the noise.

The two domains are completely different. You can't fail at driving 10% or even 1% of the time without huge problems. But if you get good code 90% of the time you can work with that. (Sounds better than some developers I've worked with.)

That's not to say that you should use AI coding agents. I'm sceptical they offer much of a productivity benefit for production code yet. But getting five nines (or whatever) of reliability is crucial for driving in traffic, but not for writing code. One is an online task that has to happen real time, the other you can apply any number of checks you want to before actually using it.

Driving is a more common skill because driving is much more obviously desirable as a skill. We do not have every young teenager counting the days until they will be allowed to code.

I believe there is a generational shift away from driving. If fewer people get drivers licenses, it will not be because driving got more difficult.

In some ways, coding may be easier for an LLM as it is just language and there are many patterns in code. However, that does not necessarily make LLMs good at design. And even being able to reflect the average is fairly bad in this case.

I think this sentence would need a huge [citation necessary] next to it :)