Notices by Tom Morris (tommorris@mastodon.social)

I started testing a popular LLM with multiple choice questions used in a professional qualification exam.

So far it's doing only a tiny smidgen better than chance alone, and way below the pass rate.

Anyone who tells you that a chatbot is gonna replace your doctor, lawyer, teacher or whatever any time soon is selling you an absolute load of baloney. None of this takes away from how good these systems are at at writing marketing copy and LinkedIn thought leader BS.

This morning, I asked a popular LLM a question about something that requires a little bit of expertise.

It precisely located the source of the information necessary to answer it... then provided paragraphs of wholly incorrect conclusions based on the correct source.

Here's a bold idea: you could have a system that gives you the source of the answer without the regurgitated incorrectness. Then rely on the human to read the original text and engage their brain.

Maybe call it a search engine.

"Sure, they can't answer questions for you, but Large Language Models will at least be useful for fixing spelling and grammar issues"

Literally just fed an LLM some text I wrote and it's changed the meaning of a number of sentences so they're not correct, invented stuff that wasn't there before, and introduced new spelling and style mistakes, including changing someone's surname from "Dove" to "Doe".

Great work, love it, definitely the future of computing—put it into everything now.

Technologists: I’m barely confident this computer is able to parse dates correctly. And don’t get me started on inkjet printers.

Politicians: they can definitely be cops.

I know, let’s use a technology that doesn’t work to solve complex social and criminal justice problems.

Yeah, this will be a recurring theme for the next decade or so.

https://www.theverge.com/2024/3/28/24114956/nyc-subway-ai-gun-detectors-evolv-technologies

@xerz @aral I've started writing a post also. I've noodled around with it a bit on Mac and have tried NixOS in a VM.

The ecosystem is quite overwhelming (and a lot of the docs are old or not great).

I'm starting by dipping a toe in and using it as a partial alternative to Homebrew, and using it for development environments and scripts... but the Nix language and Flakes etc. are somewhat mysterious. The learning curve is pretty steep.

Nix-shell shebangs are really handy.

Today, I needed to write a quick and dirty Python script to do some web page parsing and regexery, so just popped in python311 and python311Packages..

No pyproject.toml, no containers, no Docker, no venv. Run it, use result, then Nix's garbage collector will throwaway the dependencies when we're done.

https://nixos.wiki/wiki/Nix-shell_shebang

Also 24 lines of Python with httpx and bs4 that took about 20 minutes just saved me multiple hours of time-consuming work.

Horizon was raised in Parliament in 2012.

CW report in 2014. PO discontinued prosecutions in 2015. Civil litigation in 2017—2019.

Inquiry was set up in 2020. In 2021, Court of Appeal quashed a large number of convictions.

Turns out, the combined power of investigative journalism, Parliament, the courts, and public inquiries aren't enough to prompt the government to actually do anything about the biggest miscarriage of justice in English legal history—that requires an ITV drama, apparently.

A blog is a newsletter that you can run on your own server that you don’t have to share with nazis, and doesn’t have to have annoying JavaScript popups that interrupt people when they are reading things. Well worth looking into.

Amazon's "I promise you, 'asjh8a!!fWd*hasdkjfhzZ~lkjha Products and Services Ltd' is totally a real and legit Marketplace seller, if you want customer service for your broken thing, talk to them not us" retail experience combined with their "we're making our streaming service even shittier" approach to media is going to make spending less money in 2024 easier.

Inputting and editing text on a touchscreen: still the absolute worst.

They're just about usable for sending a quick text message or a social media comment but the experience of using on-screen keyboards is atrocious. And it seems to get worse not better.

It utterly baffles me that people willingly write out pages and pages of text on phones when computing devices with keyboards exist.

The Future of Life Institute's argument for how superintelligent AI will come about starts with this opener:

"AI is already superhuman at some tasks, for example numerical computations, and will clearly surpass humans in others as time goes on."

Apparently, computers adding numbers together is now "AI".

There may be good reasons to think superhuman machine intelligence might come into existence but "computers are very fast at doing arithmetic" is an incredibly rubbish argument for it.

Imagine someone said fully autonomous self-driving vehicles were inevitable. Why? Well, we've already made "superhuman" vehicles—cars, planes, boats and trains are all capable of moving much faster than humans.

Like, we may still get fully autonomous vehicles, we may not. (I've got the already existing alternative: a city with a good public transport system.)

But "a car is faster than a human" is as irrelevant to the point as how many teraflops a CPU can do is to the likelihood of AGI.

If you do not tell us how well the lady who sold you the iPhone did or the guy in Tesco who sold you a sandwich did on a scale of 1-10, and whether you'd breathlessly tell all of your friends and family about it, we will notice.

We will text you.

We will email you.

We will chase you to the ends of the earth.

We will never stop.

If you want a vision of the future, Winston, imagine turning every interaction with another human into an integer and storing it in a database, forever.

Just re-read this.

Regulators of implanted medical technology should be demanding that all source code, design docs and other technical material be lodged in escrow.

If the company ceases to make support available, make it all public.

It's one thing for movies or TV shows or video games to disappear "into the vault", but prioritising protection of completely unused IP rights over the the health and wellbeing of patients is criminal.

https://spectrum.ieee.org/bionic-eye-obsolete

Remember how a couple of years ago, there were crypto enthusiasts trying to say that artists, photographers and other creatives who didn’t get into minting NFTs would be missing out? Yeah, let’s see how that worked out.

https://petapixel.com/2023/09/20/95-of-nfts-are-worthless-report/

Ch 1 of every IP law textbook: "Copyright is vital as it gives creators a reason to create new works"

Hollywood: "We're just going to nuke this entire TV show we made because we can make more money NOT distributing entertainment than we can when we do."

Academia: "Interesting problem. Here's an article an economics professor wrote about it as part of a publicly funded research initiative which you can pay $19.99 to view for 24 hours."

Government: "we're busy making it illegal to use WhatsApp"

This is view source on the Threads homepage. Modern front end development is beyond satire at this point. In desktop Firefox with out-of-the-box uBlock Origin, it renders a completely black page.

At some point, people will rediscover the unfashionable idea that plain old HTML is pretty good if your goal is publishing bits of text on the internet. Maybe after that, if they want to make it look nice, they could look into these exciting new Cascading Style Sheets we hear so much about.

Any meaningful UX testing in 2023 needs to account for ad blockers and password managers.

If your site or app doesn’t work with popular ad blockers, or it refuses to allow logins pasted in from password managers, it’s broken.

Yes, some executive type will want to argue about this because they think ad blocking will go away or they misunderstood some now outdated infosec guidance. They’re wrong. Users use ad blockers and password managers and if your stuff doesn’t work with them, it’s broken.