Hmm! I haven't spent much time attacking yescrypt yet, but since it's based on scrypt, it has other parameters to tweak.
It looks like the spec minimum of 5 for n is implemented for common OS-level implementations?
https://www.reddit.com/r/Passwords/comments/1cd58c3/
I do not know if the r parameter can be reduced in the defaults (login.defs or whatever the equivalent is on your OS). If you can reduce r, you can drop speed further. In theory, you might be able to use mkpasswd to drop r manually and then paste that hash in?
How many milliseconds is it taking for you on that platform?