AI Generated code and the future of development.

idealizebush

What's the projects stance on AI generated code?

With tools and resources like stack overflow seeing a steep decline and companies like Anthropic making sweeping statements that developers are now using AI generated code as a canvas, eventually phasing out, is this just simply FUD?

The reason I ask is I understand there's a lot of really intelligent people who are apart of this project their opinion holds a lot of weight.

23Sha-ger

idealizebush What's the projects stance on AI generated code?

AI generated code? For a security project? You are too ambitious.
I tried to ask it to generate a sane config file for common open source projects. Came out with bullshit
settings that never existed. I then refined my query: stop hallucinating, triple check the values you give.
Made it better. Still won't trust it, it's like a reddit answer.

Delaney

idealizebush

Potentially relevant:
https://red.anthropic.com/2026/cyber-toolkits-update/

ryrona

Regardless of GrapheneOS stance on committing AI generated code to the project, there is the problem that AI coding agents are simply not good enough to aid developers in their work yet for the majority of cases, and very far away from fully replacing developers.

I have barely seen any AI code committed to serious open source project as of yet, and absolutely none to security and privacy focused project. The little AI code I have seen committed has pretty much been limited to using AI to generate optimized assembler/Vulkan/Cuda for hot code paths, which is a very tedious but well-defined translation process where the developer can trivially verify the code is correct, as any error would produce different results from the corresponding C/C++ code, and there is no memory allocations in those code paths typically.

Some AI agents can solve simple bugs themselves, but this isn't really helpful for a developer, as you have the responsibility to understand the code you commit, and verify that it is correct before committing it, and doing that pretty much involves just debugging the whole issue yourself too, at which point you could just as well just fix it yourself too. And for new features, the code produced by any AI will be of so poor quality, that it cannot be committed, and will need to be rewritten by you as the developer from scratch anyway, at which point it will no longer be AI written code. Personal experience is also that AI models today are simply not useful for security and privacy features at all, probably due to very little documentation and examples of this on the internet, and thus too little training data in that domain.

As long as AI code is not being submitted and developers are not finding AI useful as an aid in development, there isn't really much need for GrapheneOS to take any specific stance I guess.

There is also the copyright issue that I believe is not yet legally resolved.

idealizebush

Thanks for the input. I was just interested in getting peoples takes on it. There's a lot of FUD when it comes to development at the moment.

ryrona

horde

ryrona I have barely seen any AI code committed to serious open source project as of yet

KeepassXC

ryrona

ryrona I have barely seen any AI code committed to serious open source project as of yet, and absolutely none to security and privacy focused project.

horde KeepassXC

Interesting. More specifically, they used GitHub Copilot for 11 easy bug fix and test case commits between August and November 2025.

Here is a link where they write more about it, and the controversy and loss of trust in them their decision caused:
https://keepassxc.org/blog/2025-11-09-about-keepassxcs-code-quality-control/

They also disclose all AI commits there. Looking at them and the reviews, I doubt using AI improved their productivity here. But at least they tested the water to see how the community would react, and they don't seem to have made any further use of Copilot after that.

monozygote AI is here to stay. I resisted it for a long time mainly b'coz colleagues had become complacent. AI can churn out huge amount of code in no time, so folks became lazy, didn't review it beyond cursory glance, merged and deployed it.

The pace was suddenly enormous and there was a huge support from the management (they don't care about technical stuff, only $$$) because they saw it as "crushing the competitors" game. Then the breakages started, production failures etc.

There is a huge reason I think it is more interesting to see how and when serious open source projects adopts AI coding assistants than how and when closed source for-profit companies do it. For-profit companies have many reasons to invest in using AI that has nothing with productivity and usefulness today to do, including the fact that AI might boost productivity and be useful in a not too distant future, and having the company being totally unprepared and inexperienced in using AI at such a hypothetical future will make the company fall too far behind competition and become irrelevant, and that is something they must guard against by starting to use AI already now before it is boosting productivity or is being useful. Another advantage of looking specifically at serious open source projects, especially in the security and privacy sphere, is that it is the developers themselves that make the decision whether to use AI or not, and whether it is useful or not. In a for-profit company, it is management that is making that decision, and they rarely have prior experience in software development, nor even understand it, and thus makes poor decisions.

idealizebush

horde Really? Did not know this.

monozygote

I do software development for a living, so I can give my point-of-view. Background: I've done application development in Python, Rust, C++, Java and dabbled in a few other languages, for small open-source companies to Tech Giants. I DON'T consider myself extraordinary in any domain (in-fact had an imposter syndrome while working as a lead-engg for one of the tech-giants - almost everybody around me I thought was smarter lol).

AI is here to stay. I resisted it for a long time mainly b'coz colleagues had become complacent. AI can churn out huge amount of code in no time, so folks became lazy, didn't review it beyond cursory glance, merged and deployed it.

The pace was suddenly enormous and there was a huge support from the management (they don't care about technical stuff, only $$$) because they saw it as "crushing the competitors" game. Then the breakages started, production failures etc.

So now both sides, folks like me who took pride in writing code by self or after doing proper research and understanding and folks who want to "just-do-it-and-move-on" came to a middle ground - we'll use AI (claude, codex, whatever) but be very thorough with the reviews and test cases. This is working well. Junior devs can become useful much faster and AI perform better once tests break and it figures out where it went wrong etc.

AI has also got a lot better in the last few years - can take in big instructions, review-comments, search the best practice for you online etc - so it's futile to resist it. For my own personal projects I use it sparingly to prevent my coding abilities from becoming vestigial but for (at-least) a for-profit company it's suicidal to not include AI (and heavily) in software development.

Sure, for complex mathematics, kernel development etc. (and these I don't deal with), it's probably lacking just now, but will (IMO) catch up.

I see it this way: every piece of instrumentation we have built has vastly surpassed our native abilities in those areas (we can’t run faster than cars or airplanes, lift more than cranes, or see farther than telescopes, etc). AI is no different. It's nascent, flawed, hallucinates etc but it's improving and it's already helping a LOT in so many areas of software development (and other fields).

idealizebush

monozygote I agree. I work in the industry too and there's a massive push at the moment, everyone wants to be ahead of the curb with it.

guser39

monozygote I am sorry that you lack the ability to say no. Your repeated use of "futile" really shows how defeated and beaten you are. I recommend finding a different employer and counseling.

I do find it interesting to see that your shop has supposedly found a use case for GenAI. Everyone keeps saying how great it is but no one is actually demonstrating it. Has your shop published your setups so that others can audit it and use it? If not, I find it hard to believe you without proof.

monozygote

And ofc. you mileage might also heavily depend on what models you are using - gpt-5.2, opus-4.5 etc are better but cost a lot (sometime prohibitive for an individual).

Anyway, all this is, needless to say, "IMO" and as I said, I ain't a very smart chap :D

gristle-cause

If automated work is good (which most often requires human curation or at least direction), there is no reason to refuse it, if the work is bad, it should be discarded regardless of its origin, and if you accept bad job thinking it is good (or reject good job thinking it is bad) - your review process sucks (which is kind of where we at right now as humanity)

monozygote

guser39

I am sorry that you lack the ability to say no. Your repeated use of "futile" really shows how defeated and beaten you are. I recommend finding a different employer and counseling.

Lol, OK tough one, nah I'm doing fine and getting paid well, so I'll pass.

I do find it interesting to see that your shop has supposedly found a use case for GenAI. Everyone keeps saying how great it is but no one is actually demonstrating it. Has your shop published your setups so that others can audit it and use it? If not, I find it hard to believe you without proof.

Langchain/Langgraph (popular AI tooling) raised $100M in funding, OpenAI, Juni, Anthropic etc are heavily investing in coding-agents, are in breakneck competition, AWS (including Bedrock) seeing meteoric growth due to strong AI demands etc etc etc should give you a hint about it I guess?

"My shop(s)" are closed source for-profit orgs, they obviously won't publish anything for "you" to audit, but you gotta be living under a rock to ask:

Everyone keeps saying how great it is but no one is actually demonstrating it

But to each his/her own. If you haven't found it useful at all then that's fine - could be due to myriad of reasons and I don't wanna make guesses.

"Show me the evidence, show me the evidence" - Wendy Wright.

monozygote

guser39 and oh

Your repeated use of "futile" really shows [ .. ]

There's exactly one appearance of that word in my earlier post(s). But how dare the LLMs and their hallucinations!

guser39

monozygote Langchain/Langgraph (popular AI tooling) raised $100M in funding, OpenAI, Juni, Anthropic etc are heavily investing in coding-agents, are in breakneck competition, AWS (including Bedrock) seeing meteoric growth due to strong AI demands etc etc etc should give you a hint about it I guess?

Everything that I've read indicates that most of the major investing is occuring in a large incestuous circle. NVidia invests 100B in OpenAI then OpenAI invests 100B in NVIDIA. Microsoft invests 50B in OpenAI and OpenAI invests 50B in Microsoft. OpenAI is expected to run out of money in 2027. It's a house of cards with no real value beyond stock prices.

guser39

monozygote you're right. You only used the word futile once. But, your general point was that GenAiI/LLMs are not worth resisting, that it is futile and inevitable.

monozygote

ryrona That's quite accurate. Some additional points:

Much of the software I see that gets written is nothing more than being pattern-based. We do come up with more involved in-house algorithms now and then but for majority of cases the complexity is in the size of the project. If nicely designed you can break it down into smaller components with sane responsibility-boundaries and suddenly you see that most of the code is just pattern based. AI excels at these since there's not a whole lot of wiggle room for being ultra creative/doing something that's never done before. I can bet money we'll run circles round anyone not using coding agents for such tasks.
There are many factors, even programming language. I've seen cases where medium complexity tasks were done far better in Python than say in C++ (a huuugggee language with so many features). Maybe another reason why Python is now used in projects with (much) over 20K LoC absolutely regularly - trend I hadn't experienced 1st hand say 12-15 yrs back.
Coding agents are expensive. I am assuming many FOSS projects don't have that kinda budget. Codex pro is/was $200/mth, Bedrock with top notch "thinking" models are on avg $12-20 per person per day in a team. There's night and day diff between say sonnet (cheaper) and opus models. Folks without budget will obviously use low cost / free models, those are horrendous and help forming the perception "AI coding agent == Categorically a Bad thing".

Ofc it's not sentient, ofc it heavily relies on training and ofc it's a token generator at the end of the day, and might not do a stellar job in many cases but it absolutely does fantastic job in many others. It totally requires thorough review, good test cases, prompt curation etc. in any case though.

monozygote

And also, many FOSS maintainers/contibutors might be using coding-agents (but hopefully human-reviewed). They might be reluctant to admit it given the taboo surrounding the mere mention of such a thing - just look at this thread. Big guns / corporates don't give a f* (obviously) about such perceptions - they only start caring if a competitor is selling more.

If there's CLAUDE.md / AGENTS.md / GEMINI.md etc files somewhere in the repo then that's basically screaming out "we are using agents we would like our contributors to as well". However the absence of such an evidence is not an evidence of absence. So it's difficult to conclude the state of affairs in FOSS with reasonable certainty.

ryrona

monozygote And also, many FOSS maintainers/contibutors might be using coding-agents (but hopefully human-reviewed). They might be reluctant to admit it given the taboo surrounding the mere mention of such a thing - just look at this thread.

Not disclosing AI usage is erring on being unethical behavior, so I hope that is not common. I believe the open source projects that have AI policies today that are permissive towards using AI still require you to disclose if any committed code is AI made.

Let's see what the future holds. The researchers developing frontier AI models seem to believe we have hit a wall. Number of parameters and compute is no longer the limiting factor, but availability of training data is. They have already trained on the whole internet, humanity hasn't produced more online activity, code, papers or books to train on yet. Specialized small local open weight models are catching up, the difference is not that large anymore.

I think the potential for AI coding assistants is huge and very interesting, but I remain skeptical we will reach a level where we can start using them for real anytime soon, unless we see architectural improvements, and they only come about every 5 years or so.

guser39

Every major company that has supposedly gone all in on GenAI has led to massive layoffs in their workforce followed by increased outages and decreased quality. At least once a month, Amazon and Microsoft have experienced nearly day long outages in their cloud infrastructure. Nearly every monthly patch from Microsoft has had more issues than normal, including critical ones. Windows 11 is so buggy that even core functionality keeps breaking.

If GenAI and LLMs were truly so powerful and reliable, these companies would be using them internally and not selling them.

The reality is that GenAI and LLMs are so bad and lacking in potential that these companies cannot justify using their own R&D budgets to develop and mature the technology. Instead, they're socializing the cost by making everyone pay for their likely to fail aspirations. GenAI is no different from the whole VR/AR madness of a decade ago and no sooner to be a real technology than Tesla fully self driving cars, which have been a year away for a decade.

guser39

Open source projects are actively being harmed by GenAI code.

The Curl project recently had to end its bug bounty program due to GenAI harms. The sad reality is that some GenAI users were actually helpful but the vast majority were not.
Source: https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-bug-bounty/

The Apache Log4j team also mentions that they're being harmed by GenAI.

An open source project that I use and occasionally contribute to recently had one of their releases screwed up by the GenAI bot merging the wrong branches. This should have been noticed by the maintainers but they trusted it too much.

Cloudflare recently released a "production ready" project where the majority of the implementation was just TODO statements.
https://tech.lgbt/@JadedBlueEyes/115967791152135761

Not too mention how using the web has deteriorated in general. So many websites have added CAPTCHAs to prevent GenAI scraping that you can no longer just browse the web without have to prove your humanity.

ZenIsFluffy

guser39 Came here to share the Curl project's recent issues as well. Its frustrating that people are looking to make a quick buck. Combine that with how LLMs hallucinate; it has me concerned how Bug Bounty programs can continue to operate. :/