Notices by Taggart :ifin: (mttaggart@infosec.exchange), page 3

When AI pops, I'm looking forward to watching LinkedIn eat itself alive searching for a take.

Yeah so let's refocus on the real perspective here. It doesn't matter at all how good these models are at code or finding vulnerabilities if we destroy our ability to seek and share knowledge.

https://futurism.com/artificial-intelligence/google-ai-overviews-misinformation

This is the one.

Although I would have preferred it not be called "Earthset," good grief.

https://images.nasa.gov/details/art002e009288

I've been burned so many times; I've learned my lesson. You really need to read each of these things carefully if you want to understand what the researchers are concluding. Reading a news article—even worse, just the headline—is at best no information, at worse disinformation.

I am begging AI researchers trying to study human impact to get very rapidly better at methodology so I don't constantly read halfway through these papers only to find some ridiculous experiment design that will throw the conclusions into the air.

@rolle @cwebber It's very possible I would experience the tool that way were I to grip the thing a little more loosely. I wanted to maximize scrutiny in this experiment so I understood the process as granularly as possible, which absolutely contributed to the tedium.

For me though, I have to say that going faster would, while less annoying, be no more enjoyable, if that makes sense. Slow is fine as long as I'm enjoying the process. A dull process that is not creative (at least to me) is unappealing. I also suspect it's still a great way to fall into dangerous patterns. And while I understand there's six or seven different ways to hold it more correctly to avoid those patterns, they are not default, and we know where most users will land.

@elebertus @cwebber This is a lot of why I wrote it. I was tired of the "It doesn't work but I've never used it" discourse. I'm a big believer in knowing more than your opponent in a debate, and that's extra true for things you want to regulate or ban.

Ignorance about a topic undermines credibility. If we are to oppose generative models and how they're deployed, we should do so based on facts.

@cwebber I read that thread and was very surprised by Simon's recommendation to let it rip, and review changes in larger chunks. I see the rationale, but I don't see how that can do anything but increase the possibility for mistakes. Of course, additional guardrails should be in place. At what point does the energy investment in guardrails obviate the initial benefit? I'm not sure.

But I'll also add that one big reason I didn't just let it make changes autonomously is because I wanted to understand the model's process. The journey was the point. What I read in that thread was so output-focused that it missed the point entirely.

@cwebber @fhekland Ah hey, thanks!

And yeah, this question was really to get at the "We don't know," related to your point a while ago about the danger of attempting to license generated code. Basically I wanted more citations on that claim, and it sure seems like the best case scenario is "We don't know," and the worst case scenario is "Almost certainly not licensable." Either way, definitely not safe for us in open source.

I have what may be a very ignorant question: if model-generated code may not be copyrighted due to a requirement of human authorship (current US Copyright Office policy), does it therefore follow that model-generated code may not be licensed under any terms whatsoever? Meaning anything from MIT to GPLv3?

I recognize no answers here would constitute legal advice, but I would love to hear from legal experts on this.

@fhekland Hey @cwebber ☝️ this was really bothering me. If the current precedent stands, it's absolutely the case that no open source license is enforceable on generative code, as the copyright is a prerequisite for any license.

I imagine there's a test of amount still, like if most of the code is human-authored, you could still claim copyright. But for example, the tool I just made with Claude Code as an experiment? Full public domain, no terms available to me.

@fhekland

Can you license something without owning the copyright?

No, you need to own the copyright or have permission from the copyright holder to license the work.

So if no copyright is possible, no license is possible?

The Linux Foundation getting in bed with Coinbase to develop a web payments standard.

No thanks very much

https://www.linuxfoundation.org/press/linux-foundation-is-launching-the-x402-foundation-and-welcoming-the-contribution-of-the-x402-protocol

No wait, it's worse than you thought!

Membership [of the x402 governing body] will be comprised of participants from multiple verticals with initial intent and support being expressed by Adyen, Amazon Web Services, American Express, Ampersend.ai, Base, Circle, Cloudflare, Coinbase, Fiserv Merchant Solutions, Google, KakaoPay, Mastercard, Merit Systems, Microsoft, Polygon Labs, PPRO, Shopify, Sierra, Solana Foundation, Stripe, thirdweb, and Visa.

This is a list of people I don't want in charge of my money. And yes, I am forced to use some of them. Doesn't mean I trust them.

@feld I do not think letting the payment processors govern the standard could be anything but status quo

@feld Do you get the difference between being a service provider and standards maintainer? Do you get why you want a firewall between them? And why, for financial material, that firewall is extra important?

So listen.

All LLM "red teaming" is kind of a joke because of the impossibility of verifiably defending the space. There will always be a smarter mouse/jailbreak/prompt injection. But embedded application testing—that is, testing AI features once incorporated into a separate application—is uniquely pointless.

Most of the tooling to automate the drudgery of LLM red teaming assumes API access to the model/application, against which it will fire endless prompts and evaluate responses. But once the app is embedded in an application, that access is almost never available. What's left is direct application access—in other words, clicking your way to glory. Maybe you want to try to Computer Use your way to a solution, but odds are you'll just end up doing this manually. And so doing a less thorough job. And so defending even less of the possibility space.

LLMs are fundamentally insecurable, but if you only get to them once they're baked into another application, that's somehow even more the case.

@cwebber Also that! Apparently though components of it use smaller, purposed models that are not Claude? I don't know what the thinking is here, other than an unshakeable belief that generative code must be the way to do things now, and all other reasoning walks backwards from that starting position.

@cwebber Paul's reply here has been living in my head.

https://bsky.app/profile/pfrazee.com/post/3mid65y6jws2j

I fundamentally can't understand this position. Pinning all your hopes for free and open computing on "open models," a thing that doesn't meaningfully exist, is so confusing to me.

But this does appear to be dogma for them.

Claude Code's source was apparently leaked via a source map inclusion in NPM? Oh NPM, you're the gift that keeps on giving.

Here's a link to what appears to be the source: https://github.com/abubakarsiddik31/leaked-claude-code

And here is an insta-Python port of the same code: https://github.com/instructkr/claude-code

Honestly, kinda silly it wasn't open source already.