@dansup
Hey Dan. Had a look into the code, didn't see anything obvious but I am no expert in PHP/Laravel.
Fact:
The scheduled job (every 6 hours) App/Console/Commands/AccountPostCountStatUpdate.php is the culprit. Eats up all RAM (28+ GB) and gets oom killed.
Hypothesis:
Nothing changed on my side, but I did get a burst of requests from a rogue server which I blocked at Iptables level. My impression (can't be sure) is that it happens since. Could it be that status updates are too many and hog the job which gets killed before it can finish?
Proposal:
- App/Services/Account/AccountStatService::getAllPostCountIncr gets called with no limit (-1), could this induce too big a loop?
- Could we limit this loop to a number that is manageable by just setting limit = 10 000 (or whatever number) in this routine? Would this have other consequences?
Shooting in the dark here as I'm not expert here, any suggestions welcome!