The Go AI was trained by feeding it a huge number of Go games. It built a model based on w̶h̶a̶t̶ ̶h̶u̶m̶a̶n̶s̶ ̶d̶o̶. CORRECTION: KataGo is trained by playing against itself; the model input is “past AI games.”
The human beat it by doing something so obvious a human or sensible AI would never do it — so it wasn’t in the training data, so the AI didn't counter it.
(Basically, the human forms a conspicuous giant capture ring while distracting the AI with tactical battles the AI knows how to counter.)
#ai
2/