Conversation

Notices

Embed this notice
Nimda (nimda@wizard.casa)'s status on Thursday, 03-Jul-2025 08:28:56 JST Nimda

how did I not notice this all day, TL is 7 hours behind... let's see what's going on

In conversation about 2 months ago from wizard.casa permalink
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Thursday, 03-Jul-2025 08:28:52 JST silverpill
  in reply to
  
  @nimda Looks like something we should fix. Could you give me more info?
  
  In conversation about 2 months ago permalink
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Thursday, 03-Jul-2025 08:28:53 JST Nimda
  in reply to
  
  we are so back
  
  In conversation about 2 months ago permalink
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Thursday, 03-Jul-2025 08:28:53 JST Nimda
  in reply to
  
  caught this one early, newsmast.community
  removed some filters from the other day
  
  In conversation about 2 months ago permalink
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Thursday, 03-Jul-2025 08:28:54 JST Nimda
  in reply to
  
  also filtered loma.ml and bsky.brid.gy (might bring loma back if anyone misses it and it's federation stabilizes, but saw almost 400 failed to re-fetch errors in the logs since 12:00, over 2000 for kemono :aaa: )
  In conversation about 2 months ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: fed.brid.gy
    
    Bridgy Fed
  2. Domain not in remote thumbnail source whitelist: loma.ml
    
    [ˈloma] | Home
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Thursday, 03-Jul-2025 08:28:55 JST Nimda
  in reply to
  
  looks like mstdn.kemono-friends.info cloudflared their activitypub endpoints, hope rejecting will work, queue seems to be going down
  $ curl "https://mstdn.kemono-friends.info/.well-known/webfinger?resource=kita_kitsune@mstdn.kemono-friends.info" <!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title>
  In conversation about 2 months ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: i4n.kemono-friends.info
    
    ますとどんちほー
    
    Mastodonは、オープンなウェブプロトコルを採用した、自由でオープンソースなソーシャルネットワークです。電子メールのような分散型の仕組みを採っています。
  2. Untitled attachment
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Saturday, 05-Jul-2025 05:21:03 JST silverpill
  in reply to
  
  @nimda It looks like newsmast generates reposts, about once per minute, it is basically a relay. When ActivityPub server receives a repost, it attempts to retrieve the original post, and most if not all original posts are located at pubeurope.com. "Politics" channel is likely followed by many servers, so pubeurope is constantly under stress and becomes unresponsive from time to time. When it can't handle a request, it returns HTTP 502, which triggers a retry in Mitra.
  I think queue congestion is caused by a combination of a high repost rate, slow server response and 502's.
  You can try to decrease federation.fetcher_timeout, 10-20 secs is okay for most cases (but might be not enough for tor and i2p). Filtering unresponsive servers like pubeurope is also a good idea.
  In conversation about 2 months ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: pubeurope.com
    
    Mastodon
    
    The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Saturday, 05-Jul-2025 05:21:04 JST Nimda
  in reply to
  - silverpill
  @silverpill sure, I think it was mostly these:
  Jul 02 01:40:36 wizard.casa mitra[1433501]: 2025-07-02T01:40:36 mitra_activitypub::queues [WARN] failed to process activity (HTTP status server error (502 Bad Gateway) for url (https://pubeurope.com/users/fr/statuses/114780187162115427)) (attempt #1): {"@context":"https://www.w3.org/ns/activitystreams","actor":"https://newsmast.community/users/politics","cc":["https://pubeurope.com/users/fr","https://www.w3.org/ns/activitystreams#Public"],"id":"https://newsmast.community/users/politics/statuses/114780189123638405/activity","object":"https://pubeurope.com/users/fr/statuses/114780187162115427","published":"2025-07-01T21:51:34Z","to":["https://newsmast.community/users/politics/followers"],"type":"Announce"}
  I'm not sure how newsmast works and I should've just maybe filtered pubeurope.com? I see newsmast cc's other instances that throw up failures like threads.net, I'll remove the filter and see what happens
  In conversation about 2 months ago permalink
  Attachments
  1. Untitled attachment
  2. Untitled attachment
  3. No result found on File_thumbnail lookup.
    
    Barcelona
  4. Domain not in remote thumbnail source whitelist: pubeurope.com
    
    Mastodon
    
    The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!
  5. Domain not in remote thumbnail source whitelist: media.pubeurope.com
    
    France | République française (@fr@pubeurope.com)
    
    from France | République française
    
    Attached: 1 image https://www.europesays.com/fr/219146/ À l’école Longchamp, à Marseille, les parents s’inquiètent du risque d’éboulement de la chapelle mitoyenne #2025 #actu #Actualités #d'éboulement #EU #europe #FR #France #l'école #les #Longchamp... #Marseille #News #parents #ProvenceAlpesCôteD'Azur #Région #RépubliqueFrançaise #risque #s'inquiètent
  6. Domain not in remote thumbnail source whitelist: s3-eu-west-2.amazonaws.com
    
    Politics Channel (@politics@newsmast.community)
    
    105K Posts, 1 Following, 648 Followers · Welcome to the Newsmast Politics Channel. A curated feed of posts from the Fediverse, handmade by @newsmast@newmast.social, and broadcasting to Bluesky (if you've opted-in via @bsky.brid.gy)! To post to the community just add #politics to your post. You can create a list to follow this account, and exclude the posts from your home timeline. Wondering why you’ve been boosted by this account? Check out this thread: https://newsmast.social/@newsmast/111963831495789541
  7. Domain not in remote thumbnail source whitelist: media.pubeurope.com
    
    France | République française (@fr@pubeurope.com)
    
    114K Posts, 0 Following, 181 Followers · République française #France #RépubliqueFrançaise #Actualités
  8. Untitled attachment
  9. Domain not in remote thumbnail source whitelist: s3-eu-west-2.amazonaws.com
    
    Newsmast Community Feeds
    
    105K Posts, 1 Following, 648 Followers · Welcome to the Newsmast Politics Channel. A curated feed of posts from the Fediverse, handmade by @newsmast@newmast.social, and broadcasting to Bluesky (if you've opted-in via @bsky.brid.gy)! To post to the community just add #politics to your post. You can create a list to follow this account, and exclude the posts from your home timeline. Wondering why you’ve been boosted by this account? Check out this thread: https://newsmast.social/@newsmast/111963831495789541
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Saturday, 05-Jul-2025 07:44:10 JST Nimda
  in reply to
  - silverpill
  @silverpill Thanks, I looked up what the cc field was yesterday and that makes sense now. I was thinking of making a some kind of auto-filter script that monitors queue length, hits a threshold, looks at logs and filters problem instances. Maybe even have it poll the instance 5-10min if it's a 502 and remove the filter when it comes back up. Could also be possible to do something even better in mitra, if we hit a 502 move all activities that will hit that server in the queue to a secondary queue that does the slow poll until it's working and then moves everything back to the main queue? Just some thoughts, might be too much to maintain in the Mitra itself
  
  In conversation about 2 months ago permalink
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Saturday, 05-Jul-2025 07:44:10 JST silverpill
  in reply to
  
  @nimda A secondary queue for slow servers could help, but I think a more comprehensive solution is needed.
  Currently all incoming activities are processed sequentially: there is a single worker that takes jobs from the queue. We can add more workers, and thus parallelize processing (= horizontal scaling).
  
  In conversation about 2 months ago permalink
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Wednesday, 16-Jul-2025 08:46:45 JST silverpill
  in reply to
  
  @nimda You can run an additional worker in 4.6.0 with the worker command:
  mitra worker incoming-queue
  But be careful, it's experimental and may break something, so watch for ERROR and panic in the server log.
  
  In conversation about 2 months ago permalink
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Wednesday, 16-Jul-2025 08:46:46 JST Nimda
  in reply to
  - silverpill
  @silverpill more workers would help a lot!
  
  In conversation about 2 months ago permalink
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Friday, 15-Aug-2025 23:03:48 JST silverpill
  in reply to
  
  @nimda Awesome, thank you for testing.
  
  In conversation about a month ago permalink
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Friday, 15-Aug-2025 23:03:49 JST Nimda
  in reply to
  - silverpill
  @silverpill FYI spun up some workers in screen instances and it seems to help a lot
  
  In conversation about a month ago permalink
- Embed this notice
  Nimda (nimda@wizard.casa)'s status on Friday, 15-Aug-2025 23:03:50 JST Nimda
  in reply to
  - silverpill
  @silverpill awesome :acat_gangnam: I'll definitely experiment with it sometime and let you know if I run into anything :cat_cheers:
  
  In conversation about a month ago permalink

Public

Conversation

Notices

Feeds