Okay, apparently what was causing severe performance issues when loading old-ish long threads, and lately even timeline with notifs, was in fact Neetzshe's patch that preloaded scrobbles into the user profile JSON. This could've probably been solved by an index, but considering the fact I'm lazy and the frontend still loads scrobbles by hammering their separate endpoint, I just added a switch to disable returning them in every status. Having sub-second load times on timeline sure feels refreshing.
https://gitgud.io/ryonagency/pleroma/-/commit/4507b64267d7d5a767d21c091fe45bd0f42866a1