@bkuhn@ossguy I have to admit that I am pretty surprised by this post. Not in terms of being welcoming to newcomers, which is something I have advocated for and made the center of all of my FOSS work.
However, the post says the following:
> I encourage all of us in the FOSS community to welcome the new software developers who've adopted these tools, investigate their motivations, and seriously consider cautiously and carefully incorporating their workflows with ours.
While the sentence which follows acknowledges that "seasoned software developers understand the benefits and limitations of LLM-assisted coding tools", there are two big things I expected at least acknowledged:
- Many maintainers are facing *burnout* over the situation. However, I agree that addressing this in terms of norms is something we can consider - The biggest thing I am surprised to not see addressed at all is the licensing and copyright implications
@bkuhn@ossguy The surprising thing about saying "seriously consider cautiously and carefully incorporating their workflows with ours" is that it doesn't address at all my *biggest* fear: the copyright status of LLM generated contributions seems currently unsettled.
I know there's been assertions to the contrary floating around: the Supreme Court deferred to a lower court in the US. However that is not the same thing as the Supreme Court making a specific decision. And internationally, the copyright situation of output is even murkier... it will take a long time for this to settle.
Does Conservancy not think this is the case? I would be surprised if so, but perhaps you all have an interpretation that I am not currently aware of.
If there *is* concern, then we hit a serious risk: we may be seeing many contributions with legal status which has *yet to be determined* entering seasoned codebases. And this worries me a lot.
@bkuhn@ossguy There are other things I am less worried about. genAI tools used to probe for software vulnerabilities does not lead to contributions of unknown status. Same for using LLMs to explore a codebase. However, there isn't any distinction made in the post, only a "seriously consider cautiously and carefully incorporating their workflows with ours".
Does this mean Conservancy currently believes that the matter of genAI output by contemporary LLM tools is a settled matter, in terms of either a) being fully in the public domain or b) being the copyright status of the "prompter"?
@bkuhn@cwebber@ai_cases I'm confused what you mean by "dire". All LLM-emitted code being infringing would not be a "dire" outcome but the ideal one. Even if it does blow up in the faces of irresponsible maintainers who've let that infect their codebases and who now need to revert to the last non-compromised versions.
I am sympathetic to choosing to narrow a topic. However, the post, in implying that we should start accepting partially AIgen contributions, inherently pulls in the topic of whether or not that is legally safe.
Yes, I have read the previous Conservancy post about the existing cases. This partly contributes to my surprise and confusion about the post.
Acknowledging that the plan is to have continued conversations and meetings about this, I still feel it is important to lay down my current concerns, even before such a meeting. I am leaving the "quality of contributions" and many other details out of here, and instead focusing on whether of not it is *safe to accept* contributions on copyright grounds at the moment, and what the implications of thinking on that are.
@bkuhn@ossguy@richardfontana So the question is: is it safe, from a legal perspective, given the current state of uncertainty of copyright of such contributions, to encourage accepting such contributions into repositories?
Now clearly, many projects are: the Linux kernel most famously is, and their recent policy document says effectively, "You can contribute AI generated code, but the onus is on you whether or not you legally could have".
Which is not very helpful of a handwave, I would say, since few contributors are equipped to assess such a thing. I've left myself and three others addressed in this portion of the thread, and all of us *have* done licensing work, and my suspicion is, *especially* based on what's been written, that none of us could confidently project where things are going to go.
@bkuhn@ossguy@richardfontana Part of the problem here is that the AI companies have set the stage themselves. Their presumption is that it's fine to absorb effectively all open and "indie" content, and that this is entirely fair to pull into a model without any legal implications, whereas potentially yes, you may need to "license" something that looks like a Disney character. In the land of code, I also sense that Microsoft is perfectly fine with the idea that you can "copyright launder" a codebase from the GPL to perhaps the public domain, but if someone did that to their own leaked source code, they would be very upset.
Meanwhile, a friend of mine who works in films has said that he keeps hearing rumors that OpenAI would like a cut of stuff made with their stuff. We should presume tthat true.
Regardless, I'm sure everyone on this thread wants an *equitable* situation for proprietary and FOSS licensing. I'll expand on that more in a moment though.
However, it's not actually the laundering angle I am concerned with here entirely, it's whether we're turning FOSS codebases into potential legal toxic waste dumps that we will have a hell of a time cleaning up later.
The previous Conservancy post, which @bkuhn linked upthread, indicates that Conservancy does indeed consider the matter unsettled.
Current LLMs wouldn't "default to copyleft", since they also include all-rights-reserved mixed in there. If the result of output of these systems is a slurry of inputs which carry their licensing somehow, their default licensing output situation is one of a hazard.
I note that @bkuhn and @ossguy seem to be hinting at hoping a "copyleft based LLM" with all-copyleft output it a winning scenario. I'm going to state plainly: I believe that's an impossible outcome.
I would fully agree that no matter the law and legal case law passed and established, the CC0 based input LLM is clearly effectively in the public domain, or like CC0 itself, equivalent to it. This one is relatively simple.
@bkuhn@ossguy@richardfontana Regarding the one containing CC0, CC BY, and CC BY-SA, the situation is more uncertain and seems highly affected by legal outcomes in upcoming law and cases to be set. There is the possibility that indeed, the LLM is considered a slurry of inputs and this is legally acceptable, and effectively any output which is not verbatim of its inputs in some way is effectively under the public domain.
Now, of course, the problem is that we don't have to just worry about the US, we have to worry *internationally*. When considered from this angle, that FOSS is an international endeavour, this hope that things are in the public domain feels a lot dicier.
The assumption is that then this effectively leads to the output being under the terms of CC BY-SA. This is fine, great even, right?! Because effectively everything is share-alike (Bradley I don't wanna get into whether BY-SA is copyleft or something weaker). We slap CC BY-SA on the output, it's fine. Right??????
@bkuhn@ossguy@richardfontana Except, I actually believe this scenario isn't legally viable. And it's easier to understand if we scale back to the middle case.
Let's now look at the LLM trained on CC0 and CC BY. Because it's the BY aspect that makes everything complicated.
There is *NO WAY* in current LLM technology, nor I believe from studying how neural networks work, any viable computationally performant LLM, that they can track provenance. The BY clause cannot be upheld.
This isn't a theoretical concern for me; someone built another vibecoded Scheme-to-WASM-GC compiler that looks an awful lot like Spritely's own Hoot compiler in places. They didn't attribute us. They probably didn't know. But like many FOSS licenses, Apache v2 does require certain levels of attribution to be upheld. Most FOSS projects do.
You can't uphold the CC BY requirement, as far as I can tell.
@bkuhn@ossguy@richardfontana Now here is a counter-argument: how do people attribute Wikipedia? They generally just attribute Wikipedia! And people seem to be mostly fine with this.
It feels fine, when you were a contributor to the Wikipedia project.
It feels a lot less fine when you are a contributor to a specific project, to have everything just sucked up into "the generic LLM". Claude did it! Claude did it all by itself.
@bkuhn@ossguy@richardfontana If we are pushing for an *equitable* scenario for copyright output, there is only one "good outcome" in terms of copyright, and that is that everything is effectively in the public domain. The dream of having a "copyleft LLM" doesn't work.
And even if it did, there are several problems:
- Nobody is using that *now*, and contributors are facing contributions *now*, and there is legal uncertainty about accepting those contributions *right now*. - It is unlikely that the "copyleft LLM" would be very useful. The way people use these tools is conversational in a way that requires them to effectively have to be trained on the entire internet to be functional. Not just copyleft codebases.
@bkuhn@ossguy@richardfontana I say "good outcome", and I'm not saying it's an outcome I want, because "what I want" is pretty complicated here. I'm saying, it's the only one where there is the possibility of legal output from these tools that can safely be incorporated into FOSS projects *at all* that is *equitable* for both FOSS and proprietary situations.
And yup, unfortunately, that would mean copyright-laundering of FOSS codebases through LLMs would be possible to strip copyleft.
It would also mean the same for proprietary codebases.
Frankly I think it would kind of rule if we stabbed copyright in the gut that badly, but there's so much vested interest from various copyright holding corporations, I don't think we're likely to get that. Do you?
- Without knowing the legal status of accepting LLM contributions, we're potentially polluting our codebases with stuff that we are going to have a HELL of a time cleaning up later - The idea of a copyleft-only LLM is a joke and we should not rely on it - We really only have two realistic scenarios: either FOSS projects cannot accept LLM based contributions legally from an international perspective, or everything is effectively in the public domain as outputted from these machines, but at least in the latter scenario we get to weaken copyright for everyone.
That's leaving out a lot of other considerations about LLMs and the ethics of using them, which I think most of the other replies were focused on, I largely focused on the copyright implications aspects in this subthread. Because yes, I agree, it can be important to focus a conversation.
@LordCaramac@bkuhn@ossguy@richardfontana If you are talking about my personal wishes, I would agree. Personally, I perceive of FOSS as a *reaction to* allowing copyright and other intellectual restrictions laws to apply to software.
This puts me at odds with some other copyleft advocates. I see copyleft as useful because it "turns the teeth of the machine against itself". If you have copyright, then great, we will use it to have a way to force the commons to stay open.
But it would be better to have no copyright at all, and if we could give it up, I would give it up.
But it's a far-fetched dream that it could happen. Maybe it will. I am not so sure. If it truly is possible to "copyright launder" any work through an LLM, we'd be as close to it as we ever could be.
But again, whatever scenario, in my view, has to be equitable. If it's possible to do that to GPL'ed software, it's only just to be possible to do it to any proprietary software, including reverse engineering binaries.
@cwebber@bkuhn@ossguy@richardfontana I think we should just destroy copyright entirely and expand the public domain to contain everything that has ever been published. Intellectual property was a very bad idea in the first place IMHO.
@cwebber@LordCaramac@bkuhn@ossguy@richardfontana In a world without copyright (assuming no other changes), nothing would prevent people from withholding source code and attempting to restrict people’s freedom by technical means (DRM). On the other hand, it would also be entirely legal to reverse engineer everything and bypass the DRM.
Copyright should be removed, but DRM and providing binaries without source code should also be made illegal.
@cwebber@bkuhn@ossguy@richardfontana how do you launder proprietary codebases if the source isn't available? i just see this as 2 negatives since it would incentivize trade secrets
@cwebber@bkuhn@ossguy@richardfontana Worse IMHO is that we're putting FOSS as a movement at risk if we deskill everyone to the point where you either pay money to have code generated for you, or there is no code.
Are you concerned that the LLMs generate nontrivial verbatim excerpts of copyrighted works?
Or that there is a hidden "intellectual property" in the deep patterns that they use?
Say, when an LLM was trained on a file I made with an interesting loop structure, and it emits code with a similar loop structure, even if the variable names, problem domain, details, or programming language differ.
What if a court says I can demand royalties for my "IP"?
@evan@richardfontana I am saying we don't know the answer to that question, and it seems that @bkuhn and @ossguy agree that we don't know the answer to it, based on previous posts, and the lack of knowledge about what the copyright implications of LLM based contributions means that we are creating a schrodingers-licensing-timebomb for our FOSS codebases
@evan@bkuhn@ossguy@richardfontana Say for a moment that we *did* make a model which intentionally pulled in leaked source code from various proprietary codebases.
What would your opinion be on the legal-hazard state of accepting that code output? Would you consider it relatively safe from a copyright perspective?
@cwebber I think adequate compliance might be possible with good enough detection/matching tools but I don't necessarily expect such tools to be developed (let alone available to foss projects) (my assumption is that the few such tools in use today are pretty bad) @bkuhn@ossguy
@richardfontana@bkuhn@ossguy That's a problem so hard it throws the "NP complete" debate out the window in favor of something brand new. Given that these codebases have no trouble "translating" from one language's source code into another, how on *earth* could you possibly hope to build a compliance tool around that?
@zacchiro@bkuhn@ossguy@richardfontana While true, there is a big difference in that the previous scenario was someone out of compliance with what the community actually accepted as hygienic and acceptable contributions, and those contributions were relatively rare.
Saying that we don't need to worry about the risks from these tools right now from a licensing situation is different: it's advising on a path being acceptable where we *don't know* whether or not it's generally safe practice to recommend! And which most in this thread seem to agree we don't know. Even your post seems to say "it seems like it'll probably be okay and end up in our favor".
I guess I feel increasingly like I am maybe the only "oldschool FOSS licensing wonk" who cares about this, and maybe that means I should just give up.
But *damn* I can't believe it feels like when people are both saying "we don't know what the implications will be" we're also saying "so go ahead and say those patches are a-ok!"
My current answer to your "is it safe" question is to answer a slightly different question. Namely: "is it any less safe than accepting code from a random employee that claims to be submitting under a inbound=outbound regime, whereas in fact they cannot?". The latter we have been doing for decades, with limited damages to the commons.
(I *also* think the legal odds are more in our favor with AI-assisted contributions than in the previous case.)
Which maybe that means that all this stuff really is public domain, a position I am *fully willing to accept*! But I don't think it's known, and I don't think @bkuhn or @ossguy are eager to adopt that perspective
@cwebber to be clear compliance cannot somehow be built in to the LLM for reasons you stated, but ancillary tools for LLM users to reconstruct provenance exist and conceivably could be made more useful @bkuhn@ossguy
@evan@richardfontana@bkuhn@ossguy Yeah! I actually already said elsewhere in the thread I don't think we need to worry about using these tools for such scenarios from a *licensing* perspective, only when the genAI is explicitly checked into the codebase
But maybe that's wrong; I don't know. Maybe if I wrote a Person.setName() method that was in the training set, and the LLM generated an identical Person.setName() code snippet for someone else, I could claim that the code is a copyright violation, even if there were thousands of other identical and independent Person.setName() methods in the training set.
@sfoskett@evan@bkuhn@ossguy@richardfontana That outcome I am not worried about; code that's not copyrightable is considered in the public domain within the US, which means there aren't any real risks to incorporating into FOSS projects. But the Supreme Court punted on it, they didn't rule that way.
Like, the abstraction part seems like it's just summarizing components at the function, module, and program level. This is the command-line argument parser, this is the database abstraction layer, this is the logging module. LLMs are pretty good at this!
For filtration, it seems like merger or scènes à faire would also be kind of automatable, maybe with human oversight. Is there a way to make a mailing daemon without a logging module? Maybe, but it's so common that everyone does it that way. Could you have a Person class without a getter and setter for the name? Probably not?
The comparison seems tough, but I'd put an LLM to the task. "How similar are the database abstraction layers in activitypub-bot and Fedify?" Again, I'd probably want some human review, but for that code stuff LLMs are pretty good.
I consider myself an expert on this process since I learned about it 45 minutes ago, but it seems like AFC follows the hierarchical layers of modern programming-in-the-large -- statements, functions, modules, packages, program. That is the stuff that LLMs handle pretty well.
Which like, Claude is enough of a black box already but Bubble Bobble is also one of the most studied ROMs in history, so that's hard to evaluate whether it's true. You'd have to choose a less studied ROM as a test case, not Bubble Bobble, which the internet has discussed to death.
I actually think that these copyright concepts aren't particularly automatable, and even if we try, its pure arms race.
And the merger doctrine isn't the big problem here, it is the more complex analysis where merger doctrine clearly doesn't apply that needs analysis and I suspect the analysis is difficult to (even partially) automate.
@bkuhn@evan@richardfontana@ossguy Probably a ton of people here think I am anti-AI-output, and that I would be upset to find out that the chardet rewrite were legal.
Actually, I'm not! I'd be fine with the ability to copyright launder software to some degree, as long as we could do the same for proprietary software (including in binary form).
I'm concerned about whether or not we have an *equitable* situation, though. And I'm *more concerned* that we need to advise people, who are incorporating code *today*.
It missed a few things (e.g. ActivityPub relays). Then again, I have no idea how this kind of review is supposed to work. I didn't go down to the function or statement level -- that'd probably be much noisier.
Maybe chardet 6 and 7 would be a better test of the technique?
If I were going to productize this, I'd do AF passes on a huge training dataset like The Stack and generate some kind of fingerprint for each program. (Estimated cost: billions!)
I gave it a try. It's quite wordy! Claude thought that a lot of Pilgrim's work would be filtered since it was a direct port from the Mozilla C++ codebase. I pushed back that they shared the same license, and it loosened up that constraint.
Warning: if you read this document, it will get AI in you, and it will make you AI and you will become an AI-booster like me and Sam Altman. It will also burn down the rainforest.
I think you could make the case that Claude is not an uninterested party in this discussion, since Blanchard used Claude to generate the code, so maybe it's lying to cover up its tracks.
I might ask ChatGPT to give it a try, and give it some extra incentive to dig deeper because if it digs up some dirt on Claude it'd be good for business.
> “I consider myself an expert on this process since I learned about it 45 minutes ago ”
This is the second time you've made me 🤣 in this thread. Thanks for being comic relief (and I know that's not *all* you're doing, but that part is particularly helpful). Thank you!
I really don't like having anyone, including AI systems, write for me under my own name. Not least because I don't like the style and tone of ChatGPT and friends. They just write very blandly.
It wasn't till I had gone through the exercise that I realized I was doing work in a similar vein that you'd already committed to do. I hope it wasn't too monstrous.
You two have been speaking fairly collaboratively in this thread, so I'm assuming relatively synced at the moment. Since I presume the goal of "only train on copylefted [and implied, compatible] software" would not be, from your end, to erase copyleft, my assumption here is that the hope was that the output was also copylefted.
Re: “copyleft-only #LLM”: I didn't propose that. I proposed copylefting the human-modified output of LLMs.
Re: “two scenarios”: IMO you propose a false dichotomy.
I hope you come to one of #SFC's public sessions on this, as I'd be glad to talk more about it, & this discussion doesn't lend itself to online debate because it's so complex.
If that's true, wouldn't it result in a mix of incompatible copyleft licenses combined into the output? Or do you have a plan on how to deal with this?
@cwebber@bkuhn@ossguy@richardfontana Under this view it doesn't matter how the training data was licensed as it's a fair use defense. The outputs being uncopyrightable / effectively public domain allows people to claim they wrote it when it's convenient and they want to be able to copyright it as it's hard to prove if it was AI generated or human authored. And simultaneously to claim that it was the output of and LLM when they want to strip inconvenient licensing terms.
@RichardJActon The copyleft-ish hack I propose is *we* (FOSS community) assume that any output of an LLM-backed genAI system *is* copylefted (since we are pretty sure all such systems — at least those designed for software development assist — have been trained on copylefted codebases). Then, we copyleft any work that comes out of the system. The only threat is proprietary software in the training set, & the industry can't abide enforcing *that*! @cwebber@ossguy@richardfontana @evan @kees
@cwebber@bkuhn@ossguy@richardfontana I'd don't see a great way out of the copyright stripping conclusions for them without changes to the law. As I understand their defense for training on copyrighted materials - it's predicated on the models being a "transformative" and not competing directly with the original works in the market. The models themselves don't compete with the training material only their outputs do - and the LLM companies want any liability for that to be on users not them.
@cwebber@bkuhn@richardfontana@RichardJActon@ossguy@evan@kees I'm cynically assuming the new license would be more permissive. Basically GPLv3 plus explicit permission to incorporate into a "copyleft only LLM". 🤮 To normalize the idea of such a thing existing and get leverage against parties incorporating into other LLMs by saying "no that's only allowed under these terms" rather than "no you can't do that".
@dalias@bkuhn@richardfontana@RichardJActon@ossguy@evan@kees Hm, that could be clever if perceived as a permission rather than restriction. I guess it would have to be under the argument that it isn't a restriction, because it would depend on whether or not the default legislative environment provided the restriction
Taking technical debt to new highs. I'm waiting for LLMs that can tear down copyright programs from binary images and rewrite them as opensource. oh see the sparks fly then!
> I also sense that Microsoft is perfectly fine with the idea that you can "copyright launder" a codebase from the GPL to perhaps the public domain, but if someone did that to their own leaked source code, they would be very upset.
If it's leaked, that implies trade-secret law rather than copyright, no?
@chergert@bkuhn@ossguy@richardfontana Well under the more modern understanding that FOSS software licenses are both copyright and contract, I suppose the concern is also copyright and contract laundering generally ;P