Embed Notice

4. When machines reify human understanding, they just go until explicitly stopped.

5. Any misunderstanding can thus proceed arbitrarily far unchecked.

6. Software’s powerful ability to create (a) complexity and (b) simplification via layered abstractions compounds its power, but increases the odds of human misunderstanding.

7. Items 5 + 6 means that catastrophic human misunderstanding is thus more likely •and• more impactful.