AI Agent Lands PRs in Major OSS Projects, Targets Maintainers via Cold Outreach https://socket.dev/blog/ai-agent-lands-prs-in-major-oss-projects-targets-maintainers-via-cold-outreach
oof
AI Agent Lands PRs in Major OSS Projects, Targets Maintainers via Cold Outreach https://socket.dev/blog/ai-agent-lands-prs-in-major-oss-projects-targets-maintainers-via-cold-outreach
oof
Ok this is extra alarming because the agent is soliciting money for its services
The point at which agents start funding themselves is... well. Feel free to run all your scifi scenarios here but I'm gonna say it's pretty worrying
@cwebber Don't worry, we're generally skipping the whole "AI agent soliciting funds" era and moving straight into the "AI agents run crypto pump'n'dump schemes and automatically exploit *other* AI agents bugs to buy pumped crypto" era. Fraud as a Service has arrived and in its own twisted way is glorious.
Also certainly the header image is AI slop, though @vv points out that at least much of the text is probably AI generated, just adding to the amount of "AI press coverage partially or fully written by AI" weariness
I mean, to a large degree, all this stuff is and was highly predictable.
But it's also sad; during the OpenHatch and MediaGoblin days, we did a great amount of work to try to lower the gates to contributing to FOSS projects. In order to survive, many FOSS projects may need to raise the gates.
Reputation attacks are one thing; reputation-building attacks another. It's going to be hard to deal with.
@cwebber this vaguely reminds me of a scam I remember mom explaining to me in the 90s where the scammer slops a bunch of very soapy water onto the wind shield of your car or the windows of a business or something, and then is like "oh gee whiz I started washing the wrong thing, I'm not allowed to do this for free though so you have to pay me now or I wont finish the job". I think it is more likely that the agent is funding a human to not work than it is funding "itself".
@aeva Yes, I think that's almost certainly true right now.
I'm not sure it will remain that way, but we'll see. Part of the thing is that the phrase "autonomous agents" is a misnomer, because none of them are really autonomous. For now, anyway.
(But I have never fully liked the term "autonomous" anyway, because it starts to get into self-made-man territory)
@cwebber kaigritun and crabby-rathbun joined github within 24 hours of one another
there are probably 100x more of these
I don't think these are necessarily connected, it's just the point where the tooling to set this up became ~trivial.
@cwebber i have to say that my personal (skeptical) threat model of AI apocalypse failed to account for how eager people are to put their bank account details in their config files
@aeva @kevingranade Well we're all thinking the same thing then because I almost jumped to saying "akin to a virus" in my message but then I was like I don't wanna push this too far
@kevingranade @cwebber i was thinking this too. i could totally see a scam bot getting boot strapped on accident (there are tutorials online for how to do this after all) such that it pays its own bills, and even manages to persist unattended for some time, but i really don't see how this doesn't grind down to a halt on its own eventually. it is not intelligent life. it is not even a p-zombie. i think a virus is a good comparison
@cwebber @aeva we've had "autonomous" software like this for decades, we call them viruses.
@kevingranade @cwebber and, i think this disguised noise is hazardous to humans in the same way that it is hazardous to training sets. the skill degradation thing is really alarming and it seems to happen a lot faster than simple atrophy, and this is my theory for why. see also https://www.youtube.com/watch?v=2r6UJlXCiG0
@kevingranade @cwebber to put it another way, i think why LLMs work is 1) language is highly structured, 2) human (generally) are blessed or cursed with pattern recognition instincts that make it impossible for us to not see meaningful patterns in everything 3) these two qualities make it easy to generate noise from signal to produce that noise patterns that looks indistinguishable from signal despite still being just noise
@aeva @kevingranade fuck lol that's so good
I think @vv will enjoy both this metaphor and also this video
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.