It must be something else I did then, I'm not sure I do a bunch of layered defense typically for permanent sevices.
Off the top of my head, you can change the port number for ssh to something else, that usually stops basically all bots or restrict which IPs access the port via firewall, or do a VPN or ssh gateway.