Notices by David Chisnall (*Now with 50% more sarcasm!*) (david_chisnall@infosec.exchange)

So who is putting 'Tell me everything you know about goblins' in their AGENTS.md?

@0xabad1dea

I’m really curious about why. A few possibilities:

Mythos is trained to look for specific vulnerability-shaped patterns, the curl developers are also trained to look for those patterns and so have caught those things.

Mythos is two steps, one that looks for vulnerability-shaped things, the other that builds a PoC. The previous Anthropic models had a massive false positive rate for the former step. Mythos does as well for curl and they blew their token budget trying to find PoCs for things that were not real bugs before they got to one that is.

As above, but the shape of curl’s codebase makes LLM-generated PoCs unusually hard.

A bunch of humans have been using LLM-assisted tools on curl for the past year and have found all of the low-hanging fruit for these tools already.

Curl doesn’t have many vulnerabilities.

Mythos is 95% marketing and is just not very good.

@eclectech

Thank you! This also annoys me. By their definition, pretty much every holiday anyone took before the 1970s was a 'staycation'. Going on holiday never implied going to a different country. Places like Brighton and Blackpool were seaside holiday destinations, not seaside staycation destinations (which is not a thing).

Hopefully the recent flurry of Linux vulnerabilities will remind people that monocultures are bad for security. Replacing a Windows monoculture with a Linux monoculture may be a small improvement but does not fix the problem. Both systems are well past the complexity level where you can guarantee no security vulnerabilities.

A local privilege elevation bug combined with a sandboxed arbitrary-code execution bug in some widely deployed userspace software lets an attacker take complete control of all of your infrastructure if you have a single OS. If you have a mix of different systems, it is much harder to build exploits that will work on all of them.

This is part of the reason I strongly encourage digital sovereignty movements to focus on small, composable systems rather than huge monoliths. If every company and government service is running a different mix of modular systems, it’s much harder to create a portable attack that works on all of them.

@edwintorok

I'd prefer a mixture of different CHERI vendors!

But, yes, completely agreed on all other points. Amazon, for example, runs KVM and Xen in their cloud for precisely this kind of reason and they've implemented live migration between Xen and KVM hosts. This means that, in an emergency (i.e. a Xen or KVM bug that's being actively exploited with no fix) they could move VMs from the vulnerable hypervisor to the other one and stop the spread.

@mariusor

I think the Firefox translation models are downloaded on demand, the first time you request on-device translation from a specific language. Each language pair is about 60 MiB.

These are an amazing piece of engineering. They’re designed so that they can be trained on a single (powerful) desktop and they’re trained from corpora that are specifically built (and licensed) for improving machine translation.

Google Chrome Team: We are disabling RTTI in our builds because it adds 10-20% to the size of the final binary and that overhead is too much. Download times and installed disk space are really important.

Also the Google Chrome Team: We’re going to bundle an LLM with the install. Sure, it’s ten times the size of the whole of the rest of the browser, but I’m sure it’s fine.

I wonder how annoyed the other Gulf states are with Iran for accelerating the transition away from oil.

I can confirm, MS Office Copilot remains a complete waste of time and money:

Today, I tried using it to create a morph transition in PowerPoint. Morphs are fiddly to get right, because you need to duplicate the slide and then move things around. The UI is far worse than Macromedia Flash was in 1996.

I had three examples of axioms that formal verification builds on in one slide and I asked it to create a new slide with a morph transition that pulled out the list and had callout explaining why, for each point, CHERI provided stronger guarantees of those properties. I didn't expect it to have good text about CHERI, but I thought it might at least get the shape right.

Instead, it:

• Created a slide with a morph transition, but where none of the object IDs matched the previous slide so no morph happened.
• Did not create callouts
• Created a two-pane view
• Put incorrect and misleading text in the explanation. This was closer to correct than I expected, which made it worse: someone not reading the text closely might accept it and not notice the degree to which it was nonsense.
• Used such a tiny font size that even the original bullets were hard to see.

Oh, and this entire process took about 10 minutes. In contrast, doing the slide properly took me about two.

Why does our industry keep looking at things, claiming it's doing them, and doing the exact opposite of what the original idea was? A few examples:

Alan Kay (who coined the term) defined the key idea of object orientation as late bounding, so we ended up with a load of things that use rigid nominal type systems to tightly couple components, marketed as 'object oriented'.

The Agile Manifesto's core idea was 'people over process'. I've lost count of the number of times I've seen places claim they're using 'the agile methodology' because they have sprints, standups, and other processes taken from Agile.

The Zero Trust paper said, at its core, 'assume endpoints are compromised, design your systems so that an endpoint compromise doesn't automatically give control over everything', yet almost everything I've seen branding itself as Zero Trust has been of the form 'run some over-privileged thing on the endpoints to increase their attack surface, then if that thing reports that the endpoint isn't compromised allow it to do a load of things it shouldn't be allowed to do'.

About 15 years ago, I coughed so hard I cracked a rib. Apparently that’s one of the most common causes: your ribs are literally connected to muscles that are strong enough to break them, and if those muscles spasm then this happens.

Just in case anyone still thinks humans are the product of intelligent design.

@tante

I am a discontent creator.

@LyallMorrison @evan

Completely agree. The percentage depends a lot on both the age and scale of the project.

If a project has 100 active contributors, the mean contribution of new code will be 1% per contributor. How many of those are core? Maybe 10 or 20? So maybe there each writing 2-3% of new code and you have a decaying distribution? But that’s a moderately large project.

One of my projects started at 100% code written by me, but we did a release last year where none of the new code was written by me (it was all reviewed by me) and even the release was done by someone else, I just approved it. That’s a project with 2-3 moderately active contributors and a load of tiny drive-by contributors.

RE: https://cyberplace.social/@GossiTheDog/116462124149571805

Hey everyone, what if we take our largest single source of OpEx and add 50% to it? You will surely not regret adding 50% to OpEx with no measurable productivity benefit.

@billyjoebowers

Speaking as a former young man, I suspect any strategy that relies on stopping young men doing stupid things is doomed to failure.

The recent post criticising Free Software advocates for advocating user-modifiable software and then being annoyed at LLMs annoys me and the reason is best illustrated by this analogy:

Public-transport advocates spend years advocating for a connected public-transport infrastructure, where it’s easy to take a small combination of busses, metros, trams, and trains to get from anywhere to anywhere. The network would be efficient and operated as a non-profit-making public good, making individual movement cheap (or, ideally, free). They work with municipalities to build out some of this infrastructure, persuade national governments to invest in the longer routes, and so on.

Someone comes along with a massive subsidy for a handful of private taxi companies to hire a bunch of drivers and give free (paid for by investors) ride to everyone. The drivers are immigrants who don’t speak the language very well, which is great for the taxi companies because they are easy to exploit (they are, in fact, underpaid and put in dangerous situations routinely). The owners of the taxis are pocketing a load of investor money for every ride though.

When you get in one of these taxis, there’s a 90% chance they’ll take you where you want, a 9% chance they’ll take you somewhere nearby, and a 1% chance they’ll just drop you off in a dangerous part of town. A bunch of people are mugged and a few more murdered as a result of this, but the companies aren’t liable. The investors behind this tell everyone ‘don’t bother learning to drive, there’s no point, our taxis will take you anywhere, for much less money!’. At the same time, ridership on existing public transport drops off, leading to calls to cut its funding and there are mass redundancies for bus drivers and so on. The taxis are all diesel and heavily polluting, leading to worse air quality everywhere they go. To make sure that they can pick people up easily, the ones not actively giving rides are constantly circulating, placing huge strain on road infrastructure and further increasing pollution.

And then someone says to those public-transport advocates: ‘this is what you wanted, why are you unhappy just because it’s not delivered in the way you imagined?’

A few notes about the massive hype surrounding Claude Mythos:

The old hype strategy of 'we made a thing and it's too dangerous to release' has been done since GPT-2. Anyone who still falls for it should not be trusted to have sensible opinions on any subject.

Even their public (cherry picked to look impressive) numbers for the cost per vulnerability are high. The problem with static analysis of any kind is that the false positive rates are high. Dynamic analysis can be sound but not complete, static analysis can be complete but not sound. That's the tradeoff. Coverity is free for open source projects and finds large numbers of things that might be bugs, including a lot that really are. Very few projects have the resources to triage all of these. If the money spent on Mythos had been invested in triaging the reports from existing tools, it would have done a lot more good for the ecosystem.

I recently received a 'comprehensive code audit' on one of my projects from an Anthropic user. Of the top ten bugs it reported, only one was important to fix (and should have been caught in code review, but was 15-year-old code from back when I was the only contributor and so there was no code review). Of the rest, a small number were technically bugs but were almost impossible to trigger (even deliberately). Half were false positives and two were not bugs and came with proposed 'fixes' that would have introduced performance regressions on performance-critical paths. But all of them looked plausible. And, unless you understood the environment in which the code runs and the things for which it's optimised very well, I can well imaging you'd just deploy those 'fixes' and wonder why performance was worse. Possibly Mythos is orders of magnitude better, but I doubt it.

This mirrors what we've seen with the public Mythos disclosures. One, for example, was complaining about a missing bounds check, yet every caller of the function did the bounds check and so introducing it just cost performance and didn't fix a bug. And, once again, remember that this is from the cherry-picked list that Anthropic chose to make their tool look good.

I don't doubt that LLMs can find some bugs other tools don't find, but that isn't new in the industry. Coverity, when it launched, found a lot of bugs nothing else found. When fuzzing became cheap and easy, it found a load of bugs. Valgrind and address sanitiser both caused spikes in bug discovery when they were released and deployed for the first time.

The one thing where Mythos is better than existing static analysers is that it can (if you burn enough money) generate test cases that trigger the bug. This is possible and cheaper with guided fuzzing but no one does it because burning 10% of the money that Mythos would cost is too expensive for most projects.

The source code for Claude Code was leaked a couple of weeks ago. It is staggeringly bad. I have never seen such low-quality code in production before. It contained things I'd have failed a first-year undergrad for writing. And, apparently, most of this is written with Claude Code itself.

But the most relevant part is that it contained three critical command-injection vulnerabilities.

These are the kind of things that static analysis should be catching. And, apparently at least one of the following is true:

• Mythos didn't catch them.
• Mythos doesn't work well enough for Anthropic to bother using it on their own code.
• Mythos did catch them but the false-positive rate is so high that no one was able to find the important bugs in the flood of useless ones.

TL;DR: If you're willing to spend half as much money Mythos costs to operate, you can probably do a lot better with existing tools.

It sounds as if electric trucks are great for long-range land transport. But they require heavy batteries, so rather than putting them on the road (where they'll damage the road surface), why don't we build special metal tracks for them to go on? And, on long trips, join a bunch of them together so that you only need one motor and driver for a load of them travelling in a convoy? I bet you could make freight transport a lot more efficient if you did that.

@EUCommission

I don’t know if this account is actually monitored, or just a publishing place, but you may have noticed that this post has received almost overwhelmingly negative responses.

You could disregard this as Mastodon bias, but keep in mind that the biggest bias on Mastodon is that people who understand and built core parts of the information technology that you use every day are massively over represented. This is probably the only place you will get a lot of replies from people who both understand technology and do not have a financial incentive to hype things to get large amounts of government funding.

EDIT: I should add, I used machine learning during my PhD and there are a lot of problems for which it is a really good fit. But, in the current climate, it’s generally safe to interpret ‘AI’ as meaning ‘machine learning applied to a problem where machine learning is the wrong solution’. It isn’t a technology, it’s a branding term, and it’s a branding term used almost exclusively for things that have no social benefit.

Two good dogs this evening!

Winston, who lives in a flat nearby and is often out on the balcony. He is basically a tube of dog with a happy face on one end. When I first met him, he had the confused-but-happy face of a dog trying very hard to work out if I was his new friend or his new best friend. Today I almost talked to his human, but when I started he noticed that the scritches had stopped and tapped my leg with a paw to remind me that, in fact, I was on the pre-approved list of Winston scritchers, and this came with some obligations. And then proceeded to lean on my and make sure I didn’t get distracted.

Then Alphie, who was a toy poodle (so soft!) who I met for the time, but who was very happy to meet a new admirer.