Oh no. I may have accidentally found a task that an LLM is suitable for.
We have come up with what we think is a reasonable classification system for the code examples in our docs.
We have ~27,000 things that *might* be code examples.
My brain is down the rabbit hole of figuring out how to get an LLM to apply our rubric to classify our code examples.
So far, early testing = acceptable accuracy. So no need to manually audit 27,000 examples, hopefully!