Say what you will about software, but at least it’s consistent: right or wrong, at least it will follow the same instructions in the same unambiguous way.
Say what you will about the law, but at least it’s human: we constantly renegotiate it, reshape it as we live it, hot patch it as we follow it.
LLMs are neither of these things. I don’t think we have good intuitions about what they are. A lot of sloppy thinking is filling in the gaps, and investors are capitalizing on that.
/end