Conversation

Notices

1. Embed this notice
   Corné Dorrestijn (corne@cd0.nl)'s status on Monday, 09-Jan-2023 06:45:02 JST Corné Dorrestijn

   • Alex Gleason
   Upgraded from #pleroma BE to #rebased today, but haven't received any statuses for a few hours. Sending also didn't seem to work. For now I've restored a snapshot, but I am now again missing out on those nice features. Is there any documentation on how to troubleshoot Federation/why my messages are not working? @alex
   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Monday, 09-Jan-2023 06:44:43 JST Alex Gleason

     in reply to

     @corne It really depends. First thing I’d check is logs with journalctl -fu pleroma and see what’s happening there.

     Then I’d go into the database and check jobs with select count(*) from oban_jobs; Run the command several times. Are you in the hundreds, thousands, or tens of thousands? Is the number going up or down?

     If you’ve been offline for a while, you could simply have backed up jobs, and you’d need to increase your resources to catch up. This can happen anytime you’ve been offline for a while.

     If you have specific errors in your logs, we could address those.

     Rebased doesn’t change much about Pleroma in regards to federation, so I suspect something is just wrong with the new configuration.

   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Monday, 09-Jan-2023 06:47:06 JST Alex Gleason

     in reply to

     • Alex Gleason

     @corne Register on my server (or somewhere else) so you can ask me questions while yours is offline. I’ll help you figure it out.

   • Embed this notice
     corne (corne@gleasonator.com)'s status on Monday, 09-Jan-2023 07:13:26 JST corne

     in reply to

     • Alex Gleason
     @alex @corne Aah, might have been that my postgres didn't accept enough connections (20). Bumped it to 40, but my cheapo EC2 instance doesn't like that. Might need to upgrade ?
     Alex Gleason likes this.
   • Embed this notice
     corne (corne@gleasonator.com)'s status on Monday, 09-Jan-2023 07:41:17 JST corne

     in reply to

     • Alex Gleason
     • corne
     @alex @corne Yeah, will be moving to another server tomorrow. These tiny instances can't handle a lot.
     Alex Gleason likes this.
   • Embed this notice
     corne (corne@gleasonator.com)'s status on Tuesday, 10-Jan-2023 04:49:16 JST corne

     in reply to

     • Alex Gleason
     • corne
     @alex @corne The oban_jobs table is slowly getting more items, so that seems to be working. However, they are not being processed.
   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Tuesday, 10-Jan-2023 04:49:16 JST Alex Gleason

     in reply to

     • corne

     @corne @corne Has federation stopped? If the federation_incoming queue is backed up, you may need to increase the number of workers on that queue specifically.

   • Embed this notice
     corne (corne@gleasonator.com)'s status on Tuesday, 10-Jan-2023 04:49:17 JST corne

     in reply to

     • Alex Gleason
     • corne
     @alex @corne So I've moved to another server, now I've increased the number of connections to 40, but I'm again getting the too many connections errors. 40 should be enough right?
   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Tuesday, 10-Jan-2023 04:50:59 JST Alex Gleason

     in reply to

     • corne

     @corne @corne Paste the actual errors from journalctl -u pleroma --since "1 minute ago"

     You might need to increase the max_connections in Postgres itself

     What does htop look like? Are you overloaded or still have room? Make sure you’re not using too much swap.

   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Tuesday, 10-Jan-2023 04:52:05 JST Alex Gleason

     in reply to

     • Alex Gleason
     • corne

     @corne @corne Did you have editing before you made this upgrade? If the answer is no, answer this question first.

   • Embed this notice
     corne (corne@gleasonator.com)'s status on Tuesday, 10-Jan-2023 04:57:49 JST corne

     in reply to

     • Alex Gleason
     @alex @corne The logs: https://gitlab.com/-/snippets/2481922. This machine is almost Idle. I could try increasing max_connections, but 20 was enough before the switch, so I can't imagine that more than 40 is needed.
     
     > Did you have editing before you made this upgrade?
     
     What editing?
   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Tuesday, 10-Jan-2023 04:57:49 JST Alex Gleason

     in reply to

     • corne

     @corne @corne The ability to edit posts

   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Tuesday, 10-Jan-2023 04:59:35 JST Alex Gleason

     in reply to

     • corne

     @corne @corne Are you editing pool_size in prod.secret.exs, or max_connections in postgresql.conf?

     Because in postgres, you should have like 100-200 max_connections

     In prod.secret.exs, 40 is not unreasonable. I have 30 on mine.

   • Embed this notice
     Corné Dorrestijn (corne@cd0.nl)'s status on Tuesday, 10-Jan-2023 05:01:02 JST Corné Dorrestijn

     in reply to

     • Alex Gleason
     • corne
     Well, increasing the max_connections (in postgresql) seems to be working. Might be useful to add to the documentation somewhere, since Pleroma recommends 20 connections.
     Alex Gleason likes this.
   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Tuesday, 10-Jan-2023 05:02:38 JST Alex Gleason

     in reply to

     • corne

     @corne @corne They’re talking about when using the pg_tune UI

   • Embed this notice
     Corné Dorrestijn (corne@cd0.nl)'s status on Tuesday, 10-Jan-2023 05:02:40 JST Corné Dorrestijn

     in reply to

     • Alex Gleason
     • corne
     https://docs-develop.pleroma.social/backend/configuration/postgresql/#pgtune
   • Embed this notice
     Corné Dorrestijn (corne@cd0.nl)'s status on Tuesday, 10-Jan-2023 05:09:02 JST Corné Dorrestijn

     in reply to

     • Alex Gleason
     • corne
     In the Linux OTP installation guide they link to that article.
     
     > PgTune can be used to get recommended settings. Be sure to set "Number of Connections" to 20, otherwise it might produce settings hurtful to database performance.
     
     That setting will result in max_connections being 20 in the generated configuration.
     
     Thanks for helping out!
     Alex Gleason likes this.
   • Embed this notice
     Alex Gleason (alex@gleasonator.com)'s status on Tuesday, 10-Jan-2023 05:09:23 JST Alex Gleason

     in reply to

     • feld
     • corne

     @corne @corne Crazy. @feld take a look. ?

   • Embed this notice
     GNU Too (gnu2@gnusocial.jp)'s status on Saturday, 21-Jan-2023 04:03:28 JST GNU Too

     in reply to

     • Alex Gleason
     • corne
     @alex @corne personally I recommend against editing posts and "delete & redraft" Instead for a number of IOP reasons including that not every platform on the verse supports editing (your original post will be the only one ever seen by your friends on GNUSOCIAL). The speed of federation varies across the 'verse so people will still see your old unedited post after you've edited it. Some time ago I had read that DELETE is quicker that EDIT. Its less data so that makes sense.
     
     EDIT does make sense if you are juat fixing a typo AFTER you've seen folks have already intetacted with it.