Notices by David Chisnall (*Now with 50% more sarcasm!*) (david_chisnall@infosec.exchange)

The reason I get so annoyed about people pitching LLMs as a way to 'democratise programming' or as end-user programming tools is that they solve the wrong problem.

The hard part of programming is not writing code. It's unambiguously expressing your problem and desired solution. Imagine if LLMs were perfect programmers. All you have to do is write a requirements document and they turn it into a working program. Amazing, right? Well, not if you've ever seen what most people write in a requirements document or seen the output when a team of good programmers works from a requirements document.

The most popular end-user programming language in the world (and, by extension, the most popular programming language), with over a billion users, is the Calc language that is embedded in Excel. It is not popular because it's a good language. Calc is a terrible programming language by pretty much any metric. It's popular because Excel (which is also a terrible spreadsheet, but that's a different rant) is basically a visual debugger and a reactive programming environment. Every temporary value in an Excel program is inspectable and it's trivial to write additional debug expressions that are automatically updated when the values that they're observing change.

Much as I detest it as a spreadsheet, Excel is probably the best debugger that I have ever used, including Lisp and Smalltalk.

The thing that makes end-user programming easy in Excel is not that it's easy to write code, it's that it's easy to see what the code is doing and understand why it's doing the wrong thing. If you replace this with an LLM that generates Python, and the Python program is wrong, how does a normal non-Python-programming human debug it? They try asking the LLM, but it doesn't actually understand the Python so it will often send them down odd rabbit holes. In contrast, every intermediate step in an Excel / Calc program is visible. Every single intermediate value is introspectable. Adding extra sanity checks (such as 'does money leaving the account equal the money paid to suppliers?') is trivial.

If you want to democratise programming, build better debuggers, don't build tools that rapidly generate code that's hard to debug.

@jimcarroll Ah yes, Canadians, famed worldwide for their rude and unhelpful behaviour.

@ChrisMayLA6 @GeofCox Access to treatment for diabetes is the best case study I’ve seen for this. Cost cutting there increased the number of amputations and of prosthetics and long-term disability support needed and offset the cost savings by over and order of magnitude. But the two are in totally different parts of the NHS budget and so the people responsible for the cost savings looked good to their management.

I really like capability systems, but 'capability' is a terrible name.

In normal English, a capability is something intrinsic. If you have the capability to run a four-minute mile, it's something that you can do. You don't need some token to enable you to do it.

In a capability system, holding a capability doesn't grant you the ability to do the thing implicitly, it requires you to present the authorising capability when you try the action. This is one of the core advantages of capability systems over other kinds of access control. They respect the principle of intentional use. It's not enough that you have a capability to do a thing, you must use the correct capability when you try to do the thing. This eliminates a whole set of possible confused-deputy attacks.

Capabilities are more like inventory items in an adventure game. Just having them in the inventory doesn't let you solve a puzzle, you must use the correct inventory item on the correct object to solve the puzzle.

I can't think of a better word. 'Tool' might work (except that it's almost as bad as that time some French people named a theorem prover). Saying 'I don't have the right tool to accomplish this task' makes sense in English as a 'I need to hold this thing and use it correctly', whereas 'I don't have the capability to accomplish this task' sounds like you're talking about ambient authority.

Are there better words? Maybe something in another language?

@DemocracyMattersALot

the app is known to have been breached by Russian and Belarusian hackers

That is misleading. If you scanned a QR code in Signal, it would let you link a device that would then see all messages sent to you until you noticed and unlinked that device.

#Signal is a long way from perfect, but it has one key benefit: It is very easy for people to adopt. Download the app, follow a linear flow, start talking to existing contacts. In particular:

• It reuses an existing identifier (COI also does this, though doesn’t have a discovery protocol so you can’t tell which users will support the upgraded mode without trying to message them).
• The security of your communication does not depend on the instance you pick (you can’t pick an instance and you can’t choose to run one, which I consider a weakness, but most alternatives at least leak metadata to the person who runs your server and to the servers that you communicate with).
• It doesn’t ever fall back to an insecure mode. This eliminates downgrade attacks. OMEMO over XMPP and COI will both fall back to unencrypted modes and rely on UI patterns for users to detect this (remember how well the SSL padlock worked for this?)
• It reliably syncs keys across devices, so phone, tablet, and desktop clients can all see your messages and you can transparently move between them.
• Encrypted multi-party video chats just work. Now that chat links works you can just stick them in a calendar invitation and switch to using Signal for meetings instead of proprietary systems.

There’s a longer list of things that I would like them to fix, but that’s the set of things I need a chat program to support to be something I’d recommend to non-geeks. Being able to host your own server is nice. Needing to host your own server for security is a deal breaker.

Yay, after only seven years, the latest update to Microsoft Teams lets you past an email address into the 'recipients' box in the make-a-meeting UI and doesn't split it into {first} {second} {email@address} and treat them all as separate invalid email addresses (yes, even the email address, because it kept the angle brackets in the token that it parses. And, yes, it did split between spaces for things in quotes, but it dropped the quotes).

Now, if you paste "Some person" <email@address>, you get that person as an invitee, not Some, person and <email@address>.

Honestly, if I didn't have first-hand knowledge that people in the Office org at Microsoft use their products, I wouldn't believe it.

@GossiTheDog Does Mirage get better? I find most of this series starts slowly, but I got about an hour into Mirage and then went and replayed Odyssey instead (the Atlantis story lines in Odyssey remain some of my favourite, mostly because I like stealth murdering statues).

@PhilGastwirth @SecurityWriter The USA, which insists that everyone else respects its IP protections, has never signed any of the treaties that protect product origins (AOC and so on). As a result, you can call anything champagne in the US. I've even seen a fizzy soft drink sold as 'champagne' there.

I have lost count of the number of people at Embedded World who have asked me ’what is memory safety?'

If anyone is wondering how embedded security is going...

#EmbeddedWorld #MemorySafety #CHERI

I turned down an invitation to serve on a PC this morning. Computer Science publication has been a bit of a mess for ages but AI slop had pushed it over the edge. I no longer consider it a worthwhile use of my time to review papers that may have no relation to any real system. Unless a venue has a process to ensure that papers are written by real people and describe real research, I won't waste my time reviewing fake papers. I don't think the current double blind process can work in a world where writing a slop paper takes less effort than reviewing one. I think moving to a post-publication review model is the only workable solution. Put the paper, code, and everything else necessary to reproduce the work online and then ask for reviews.

@SrRochardBunson @feld @redstateinsurgents

it uses encypted email in the background & nothing is stored on anyone's server in any country.

These two statements are opposite. If it uses email, then messages are stored on a mail server. Even if it’s encrypted, the sender and receiver must be plaintext and are visible to the operators of both the sending and receiving mail servers, which operate in at least one country.

If you use your own mail server, then it’s easy for a third party to just watch the (encrypted) network traffic to know who you are talking to. If you use one of the big ones such as GMail or Outlook Online then the operator of that server sees all of your connections. If one of your correspondents uses a big mail server then the operator can see your connection to their users and build at least a partial social graph.

This kind of metadata is incredibly valuable to both advertisers and more malicious users. Facebook bought WhatsApp harvest this data, because it was sufficiently valuable to them even in the presence of end to end encryption. It’s the same thing that nation-state dragnets collect.

The Signal protocol was designed to make it hard for even the operators of the server to collect this information.

@Jyoti It really astonishes me how PE teachers manage to take games, activities explicitly created to be fun, and make them the least enjoyable part of school.

We used to get a load of trainee PE teachers and they’d all been taught to say ‘would you rather do maths?’ when people weren’t following their instructions. They never seemed to know what to do when the unanimous response was ‘yes!’

@sybren @signalapp Is there any public data on how much Signal costs to operate? WhatsApp was making a profit on a $1/year subscription fee.

Much as I like #Signal, threatening to leave a country if they pass laws banning end to end encryption demonstrates a weakness in the system. The Signal Foundation should not have the technical ability to 'leave' a country. At worst, they should be able to stop receiving donations from people in that country.

@simon_brooke @aral 64 bits is not enough for a vaguely secure system. Cookies are not under the server's control. If you are using 56-bit user ID and that's there as raw data trusted on the host, then it's easy for an attacker on any site with a non-trivial number of users to just send random numbers in the cookie and hijack existing sessions. Remember the birthday paradox.

128 bits is probably fine, since that would let you store an ephemeral UUID.

That said, a 128-bit limit require that you store all state on the server. No equivalent of local storage means that things like disconnected operation for web apps is not feasible. Maybe that's a goal, but I'd generally prefer to minimise the amount of server state (which is not under my control) rather than minimise the amount of client state (which is mine to inspect).

@futurebird @CptSuperlative @emilymbender Summarisation is the one I’d be most nervous about because creating a summary is hard: it requires understanding the content and knowing which parts are relevant in a given context, which is why LLMs tend to be awful at it. They don’t understand the content, which is how you get news summaries that get the subject and object the wrong way around in a murder. They don’t know what is important, which is how you get email summaries that contain a scam message and strip all of the markers that would make it obvious that the message is a scam.

If you’re going to read all of them and are just picking an order, that’s probably fine. The worst that a bad summary can do is make you read them in the wrong order and that’s not really a problem.

@nieldk @GossiTheDog That tells you that the sending domain has the correct entries, it doesn’t check that the receiver does the right thing for invalid emails.

For example, when I was at Microsoft, all of the right DNS entries were configured for the microsoft.com domain, but if I sent myself an email via my own mail server claiming to be from a MS email address (invalid sender according to SPF, invalid DKIM signature) it was still delivered (and Exchange helpfully stripped off all of the SMTP headers that would make it obvious that it came from an invalid sender).

@GossiTheDog What’s the betting that they don’t check SPF and DMARC signatures and so just accept emails from president@whitehouse.gov as if they are legitimate?

Steven Spielberg said, after he made Schindler's List, that it was partly an apology for portraying Nazis in the Indiana Jones franchise as less evil than they really were. I think Hollywood has been guilty of something far worse in recent decades: portraying Nazis as competent.

Germany wasn't a country run by Nazis that happened to lose the Second World War, it was a country that lost because it was run by Nazis.

Take a look at the names of the folks that worked on the Manhattan Project. See how many of them are German? Several of them worked on weapons in the First World War, for the Germans. Germany had a huge lead in developing a nuclear weapon in the '30s, but removed people who weren't Nazi enough from positions of authority in fields related to weapons research. A load of their best scientists were on the various lists that would end up on death camps and managed to leave (others didn't, and died). When you start by saying 'only people from this arbitrary subset of the population based on race / gender / religion / sexuality / whatever may contribute to our society', you won't get the best people.

Hitler maintained control by promoting people based on their personal loyalty to him, not based on their competence. He ensured communication flowed through him and made parallel agencies compete, directing their effort against each other rather than towards shared goals, to avoid any becoming powerful enough to challenge his power structures.

Hitler was almost responsible for most of the German army being wiped out early on in the Second World War because he had an exaggerated opinion of his own ability. The only reason it wasn't was that allied commanders didn't believe anyone could be that stupid and assumed it was a trap (it wasn't, he really was that stupid). He then decided to invade Russia in the winter (which worked so well for Napoleon) against the advice of any of the people who paid attention in school, which was one of the key turning points in the war.

Don't let the smart uniforms fool you. They were not competent people who lost as a result of circumstances beyond their control. They were people with an ideology that was ultimately self defeating in the long term. And, if people with the same views are in positions of power again, they will fail in the same way. The problem is not that they might succeed, it's that they have a habit of taking a lot of other people with them on their way to defeat.