To be clear up front, these are my personal opinions right now, and while they will influence any future policy I have a hand in writing, they are not the position of the Biopython Project or the Open Bioinformatics Foundation where hold a leadership role.
So, it took longer than I expected, but we've had our first Biopython pull requests or proposals openly using generative AI. On the bright side, they are not trivial drive by slop in some perversely incentivized scheme to get a T-shirt or pad a CV. But I still have qualms, lots of them!
Ethics
There are at least two generic ethical issues with generative AI coding (not specific to the projects I work with), moral and environmental. Most of these systems have been trained with data in what might best be described as a legally gray area in copyright terms. Even light touch Open Source licensees like the MIT and BSD licenses have a requirement to preserve copyright statements, let alone the reciprocal sharing expectations with the GPL. Even if judged legal, how their work has been used to train these systems, is counter to the expectations of many of the authors sharing their code publicly.
Next, the data centers used to run these models use a substantial amount of electricity and water for cooling, and collecting the training data (imposing costs on third party servers) and training their large language models (LLMs) takes an even larger toll. This is not something we as a society should encourage, and is counter the global pressing need to reduce energy consumption, and exacerbates the expected rise in water-driven conflicts from climate change.
Legality
This is probably a non issue, but what if the copyright situation does change (in at least some parts of the world), and hundreds or thousands of open source projects are deemed to be incorporating unlicensed copyrighted works? Chaos, which of course the lawyers or the big AI firms seek to avoid as this would destroy their business... but when the AI bubble breaks will there be as many well paid lawyers on that side of the argument? This is I accept fear mongering, but it is at the back of my mind as a worrisome potential liability.
Quality
Many experts have written about the fact that while gen AI may be able to get a prototype up and running, the generic code can be brittle and buggy (including security bugs). This can be mitigated in the hands of an experienced developer guiding the system and essentially using it for the repetitive grunt work in a task. I can see the attraction in a language with a lot of boilerplate code. Here's an example from Mitchell Hashimomo vibe-coding a non-trivial Ghostty terminal feature. But in the hands of even a well meaning novice, you can get slop.
Maintenance
For a (potentially) long running Open Source Project, long term maintenance is already a real problem. Biopython is nearly 25 years old, and has seen contributors, leaders, and maintainers come and go. My own tenure has been unusually long. Most contributors make one or two fixes (often during their PhD) and move on - partly driven by the chopping and changing of projects on a typical post-doctoral career.
Sometimes the original contributor stays involved long enough to see their new functionality mature and survive real world usage. But with generative AI, no one wrote the code, and the human most involved may themselves not fully understand the code proposed. There is also the real concern that if generative AI is used to handle many of the easier changes, it makes it harder for newcomers to contribute to a project and learn - a real long term issue for programming generally.
With a Python project like Biopython, we have some code in C, but very few maintainers competent and comfortable reviewing and maintaining that. This means vibe-coded C contributions have an extra-high barrier to acceptance. And this is even worse with suggestions to add another language like Rust which would put additional complexity on the maintenance of the build system, continuous integration, and release process. The only way I see that happening is if a regular contributor/maintainer themselves was to push for adding library code in a new language, and take on that maintenance themselves. But even then, I would be wary of the bus factor with everything dependent on one person.
Saying no
One of the things you have to learn an maintainer of any software package (near solo hobby project, or large collaborative long term community project) is to be willing and able to say no to a feature request or pull request. eg Knowing when to say 'no' by Jon Parise (and many other pieces). Right now, I still lean very much to saying no any PR using generative AI.
No comments:
Post a Comment