So it might be nice to use the setup of someone who has figured this out after all.
On https://wiki.openstreetmap.org/wiki/OSM_on_Paper, I found a web service that comes really close to what I'd like:
Most importantly, you can set reasonable map scales like 1:10000, which some other tools didn't allow!
It has a maximum paper size, and some glitches when you come close to it, but I generated two overlapping images and stitched them together in GIMP…