Engineers need to do better

Posted Jun 4, 2025 14:26 UTC (Wed) by willy (subscriber, #9762)
Parent article: The importance of free software to science

Most working scientists are not computer experts (those who are frequently get recruited into industry and stop publishing). This means that most software written by scientists is of poor quality and does not use best practices which will let it run on future substrates.

I found some code which had been written against Python 2.6 and would have needed substantial changes to make it work with 2.7. Part of that was using a library which was no longer available. And I'm no Python expert, so I just gave up.

You're right that open source software gives us an advantage, but we have to have a better legacy story than this! Whether that's preserving digital artifacts better or having a better backward compatibility story or something else ...

Engineers need to do better

Posted Jun 4, 2025 16:07 UTC (Wed) by fraetor (subscriber, #161147) [Link] (1 responses)

This is a notable issue in research, but one for which there is progress, albeit gradually. The role of a Research Software Engineer (RSE) is becoming more common to support technical development in science and there is a growing body of practice aiming to improve the state of software. This practice includes both traditional development "best practice", as well as training of scientists and pushing for the software lifecycle to be included in grants and proposals. [1]

Software is an area where university based researchers often struggle more than their industry counterparts, largely due to the short term nature of university funding and contracts, and a focus on publication output for promotion, etc. Over the past few years a number of UK universities have established a central pool of RSEs, often employed on a permanent basis, to mitigate this.

However, while there is a lot of focus around reproducibility, especially in the context of FAIR [2], it does seem that a lot of the effort is going towards freezing all the dependencies and effectively reproducing the original environment, whether through conda environments, containers, or VMs. I guess it is the difference between single paper analytics, and creating a reusable analytics tool.

Software is more essential to science than ever before, so this is definitely an area to keep on improving.

[1]: J. Cohen, D. S. Katz, M. Barker, N. Chue Hong, R. Haines and C. Jay, "The Four Pillars of Research Software Engineering," in IEEE Software, vol. 38, no. 1, pp. 97-105, Jan.-Feb. 2021, https://doi.org/10.1109/MS.2020.2973362
[2]: Wilkinson, M., Dumontier, M., Aalbersberg, I. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3, 160018 (2016). https://doi.org/10.1038/sdata.2016.18

Engineers need to do better

Posted Jun 4, 2025 22:01 UTC (Wed) by kleptog (subscriber, #1183) [Link]

So there's a name for that now: Research Software Engineer.

The projects at $DAYJOB I find the most fun are when I'm given a pile of code written by some researcher or analyst to solve a problem to turn it into something usable. The joy on their faces when you spend an afternoon restructuring their code to produce something more readable and reliable and completes in a fraction of the time is priceless.

Engineers need to do better

Posted Jun 4, 2025 17:21 UTC (Wed) by fenncruz (subscriber, #81417) [Link]

This means that most software written by scientists is of poor quality and does not use best practices which will let it run on future substrates.

As someone who has made the move between academia and industry I can assure you that people can and do write bad code in any ~~language~~ industry.