Notices by Simon Willison (simon@fedi.simonwillison.net), page 8

If you haven't been following the wild news out of OpenAI (short version: the board of the controlling non-profit fired CEO Sam Altman yesterday) this piece by @benjedwards is a really great summary of everything that's known so far https://arstechnica.com/information-technology/2023/11/report-sutskever-led-board-coup-at-openai-that-ousted-altman-over-ai-safety-concerns/

@HistoPol @annaleen the original ChatGPT turned out to be a lot less prone to wild vengeful outbursts than whatever model it was that they plugged into Bing - it's a pretty interesting demo of how well the safety features of ChatGPT (which Bing seems not to have) have held up

I presented a ten minute segment at GitHub Universe on Wednesday: **Financial sustainability for open source projects**.

Here's the video, slides and annotated transcript: https://simonwillison.net/2023/Nov/10/universe/

Open source maintainers are often good at writing code but not good at asking for money

Companies aren't very good at giving money away, but they absolutely know how to hire consultants - and they often have a training budget set aside already

Spend that money on maintainers!

My closing suggestion was to support open source projects you depend on by reaching out to their maintainers and offering to pay them money to speak to your team! Ask them for an hour over Zoom or similar.

Here's a clip with this idea in more detail

New LLM paper highlighting quite how weird and ridiculous these things are https://arxiv.org/abs/2307.11760

Adding "it's important to my career" can produce better results, across every model they tested!

@feditips @Snowshadow @mhoye I think it's this bug here https://github.com/mastodon/mastodon/issues/24676

@feditips @Snowshadow @mhoye that's the bug: it's definitely a thread!

You can see that it's a thread if you follow this link to the second post in that thread: https://mastodon.social/@mhoye/111336017090790537

OK, I have a somewhat baffling (to me) Mastodon question. How do I link to a thread?

I want to link to a fantastic thread by @mhoye - but if I link to the first post in that thread - https://mastodon.social/@mhoye/111335603309582734 - I get a page with a single post on it and other people's replies, with no indication it's part of a larger thread from the same author

Am I missing something here?

New release of jq - the first in five years! Notes here: https://simonwillison.net/2023/Oct/2/jq-17/

Here's a thought-provoking take:

"We already know one major effect of AI on the skills distribution: AI acts as a skills leveler for a huge range of professional work. If you were in the bottom half of the skill distribution for writing, idea generation, analyses, or any of a number of other professional tasks, you will likely find that, with the help of AI, you have become quite good."

Ethan Mollick - https://www.oneusefulthing.org/p/everyone-is-above-average

For me this is the single most positive potential future outcome of the current leap forward in terms of AI capabilities: tools that helps human beings level up, as opposed to replacing them

A big question here (as Ethan outlines further down in that post) is whether organizations will chose to do more with their existing teams, or will downsize them to try to achieve the same with less people

Early hints we're getting from stock photography, translation & illustration don't look great so far

I'm particularly interested in translation (between human languages) because that's been quietly impacted by leaps forward automation for 5-10 years at this point

I'd love to see some credible studies on what the impact of that has been on the human translator industry: has there been mass unemployment, or are existing translators getting more work but it's more "clean up this AI-generated version", or is it a mixture of both?

@band Oops, that should be "llm embed-multi" - will update, thanks!

Big new release of my LLM CLI tool and Python library for working with Large Language models (Llama 2, GPT-4 etc)

LLM 0.9 adds support for embedding models, installed via plugins

If you aren't familiar with embeddings I have a very detailed explanation of what they can do and how you can use them here:

https://simonwillison.net/2023/Sep/4/llm-embeddings/

Here's a fun example of something you can now do with LLM: search for every README.md file in your home directory and store embeddings for all of them in a collection called "readmes":
```
llm embed readmes \
--model sentence-transformers/all-MiniLM-L6-v2 \
--files ~/ '**/README.md'
```
Then run a similarity search for "sqlite" like this:
```
llm similar readmes -c sqlite
```

Leaked Google document: “We Have No Moat, And Neither Does OpenAI”

The most interesting thing I've read recently about LLMs - a purportedly leaked document from a researcher at Google talking about the huge strategic impact open source models are having
https://simonwillison.net/2023/May/4/no-moat/

@jimgar the ethics of this stuff is incredibly complicated

I'm very optimistic about the models being trained on the RedPajama data - there's one out already and evidently more to follow very shortly https://simonwillison.net/tags/redpajama/

@resing @jimgar I'm not convinced it's possible to train a usable LLM without including copyrighted material in they raw pretraining data

As such, personally think it's a necessary evil to avoid a monopoly on LLM technology belonging to organizations that are willing to train against crawler data

@resing @jimgar I'm saying I'm not sure it's possible to build a useful LLM without including copyrighted data in the training set

The ethics of this entire field are incredibly murky - I wrote about that last year https://simonwillison.net/2022/Aug/29/stable-diffusion/#ai-vegan