Conversation

Notices

Embed this notice
Emanuele (emanuele@m.divita.eu)'s status on Saturday, 26-Nov-2022 04:38:14 JST Emanuele
- Aral Balkan
- Masto.host
@aral @mastohost do you know you can periodically prune the image cache without breaking anything? The last release allows to enable it by the administration panel too.
Aral, I have to say… I was not expecting to see you using #AWS!

In conversation Saturday, 26-Nov-2022 04:38:14 JST from m.divita.eu permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 04:37:15 JST Masto.host
  in reply to
  - Aral Balkan
  - Ryuno-Ki
  @RyunoKi @emanuele @aral there is already automated cache cleaning in place. As explained here https://masto.host/re-mastodon-media-storage/ or the numbers would be much higher.
  But Aral can use more aggressive limits in the admin interface. For media it’s fine but for content it can lead to unwanted consequences as I mention here https://github.com/mastodon/mastodon/discussions/19260
  In conversation Saturday, 26-Nov-2022 04:37:15 JST permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: masto.host
    
    RE: Mastodon media storage
    
    from Hugo Gameiro
    
    To clarify, when you or another user in your server follows a remote user, from that moment on, all new posts from that remote user start to federate with your server. By federate, you can…
  2. Domain not in remote thumbnail source whitelist: opengraph.githubassets.com
    
    RE: retention policy for cached content and media · Discussion #19260 · mastodon/mastodon
    
    The recent PR #19232 adding a retention policy for cached content and media can significantly reduce the resources used by Mastodon. In particular, the storage needs. This is something that admins ...
- Embed this notice
  Ryuno-Ki (ryunoki@layer8.space)'s status on Saturday, 26-Nov-2022 04:37:16 JST Ryuno-Ki
  in reply to
  - Aral Balkan
  - Masto.host
  @emanuele
  This.
  @aral @mastohost
  
  In conversation Saturday, 26-Nov-2022 04:37:16 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 04:41:08 JST Masto.host
  in reply to
  - Aral Balkan
  - Ryuno-Ki
  @RyunoKi @emanuele @aral Still what can actually help is this https://github.com/mastodon/mastodon/pull/21066 because currently there is no way to remove avatars and headers. And everyone that ever federated with a server will have their avatar and header cached forever.
  In conversation Saturday, 26-Nov-2022 04:41:08 JST permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: opengraph.githubassets.com
    
    Implement account media prune and fetch command by dunkelstern · Pull Request #21066 · mastodon/mastodon
    
    This implements two new sub-commands for tootctl media: prune-profile-media to remove old avatar and header images fetch-profile-media to download missing avatar and header images basically undoin...
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 20:24:11 JST Masto.host
  in reply to
  - Aral Balkan
  - Ryuno-Ki
  @aral No idea, it was just a suggestion but AFAIK nobody picked up on it yet.
  @RyunoKi @emanuele
  
  In conversation Saturday, 26-Nov-2022 20:24:11 JST permalink
- Embed this notice
  Aral Balkan (aral@mastodon.ar.al)'s status on Saturday, 26-Nov-2022 20:24:12 JST Aral Balkan
  in reply to
  - Masto.host
  - Ryuno-Ki
  @mastohost Do you know if they’re going to implement your reference counting suggestion. Because that sounds optimal.
  @RyunoKi @emanuele
  
  In conversation Saturday, 26-Nov-2022 20:24:12 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 20:25:24 JST Masto.host
  in reply to
  - Aral Balkan
  - Ryuno-Ki
  @aral Yep, I was following it. If you want, I can run a media stats on your media for you to see how much you are using for each (avatars, headers, preview_cards, attachments, etc). Just ping me and I run that for you.
  @RyunoKi @emanuele
  
  In conversation Saturday, 26-Nov-2022 20:25:24 JST permalink
- Embed this notice
  Aral Balkan (aral@mastodon.ar.al)'s status on Saturday, 26-Nov-2022 20:25:26 JST Aral Balkan
  in reply to
  - Masto.host
  - Ryuno-Ki
  @mastohost Ah, and it looks like that’s closed for now :(
  @RyunoKi @emanuele
  
  In conversation Saturday, 26-Nov-2022 20:25:26 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 20:44:53 JST Masto.host
  in reply to
  - Timothée Goguely
  @timotheegoguely I have just emailed you the results.
  Adding it to Masto.host dashboard is something that's in my todo list. Sorry, not time for everything but slowing more will be added there.
  
  In conversation Saturday, 26-Nov-2022 20:44:53 JST permalink
- Embed this notice
  Timothée Goguely (timotheegoguely@mastodon.design)'s status on Saturday, 26-Nov-2022 20:44:54 JST Timothée Goguely
  in reply to
  - Masto.host
  @mastohost I'm also interested if you could run a media stats on mastodon.design I think it would be an awesome feature to add to masto.host dashboard!
  
  In conversation Saturday, 26-Nov-2022 20:44:54 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 21:01:59 JST Masto.host
  in reply to
  - Aral Balkan
  - Ryuno-Ki
  @aral No problem at all. Emailed you the results.
  @RyunoKi @emanuele
  
  In conversation Saturday, 26-Nov-2022 21:01:59 JST permalink
- Embed this notice
  Aral Balkan (aral@mastodon.ar.al)'s status on Saturday, 26-Nov-2022 21:02:00 JST Aral Balkan
  in reply to
  - Masto.host
  - Ryuno-Ki
  @mastohost Ooh, that’d be interesting to know. But only if it’s no trouble, Hugo and if/when it’s convenient. I know you’ve got your hands full with the influx of new people.
  @RyunoKi @emanuele
  
  In conversation Saturday, 26-Nov-2022 21:02:00 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 21:06:32 JST Masto.host
  in reply to
  - Aral Balkan
  - Ryuno-Ki
  @aral Yep, the other items are automatically cache deleted using `tootctl` with the number of days mentioned on https://masto.host/re-mastodon-media-storage/
  But `tootctl` does not allow to clear the Avatars/Headers cache and they are cached indefinitely for everyone that has ever federated with your server.
  @RyunoKi @emanuele
  In conversation Saturday, 26-Nov-2022 21:06:32 JST permalink
  Attachments
  1. Untitled attachment
- Embed this notice
  Aral Balkan (aral@mastodon.ar.al)'s status on Saturday, 26-Nov-2022 21:06:34 JST Aral Balkan
  in reply to
  - Masto.host
  - Ryuno-Ki
  @mastohost Ah, thanks, appreciate it.
  So >60GB are headers and avatars… wow.
  @RyunoKi @emanuele
  In conversation Saturday, 26-Nov-2022 21:06:34 JST permalink
  Attachments
  1. Untitled attachment
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 21:37:46 JST Masto.host
  in reply to
  @jwildeboer I agree. This is something that Mastodon needs to fix and someone is already working on it https://github.com/mastodon/mastodon/pull/21066 but it's still having trouble getting it to work properly.
  Hopefully this will be solved soon.
  @fiftyfourpark @emanuele @RyunoKi @aral
  In conversation Saturday, 26-Nov-2022 21:37:46 JST permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: opengraph.githubassets.com
    
    Implement account media prune and fetch command by dunkelstern · Pull Request #21066 · mastodon/mastodon
    
    This implements two new sub-commands for tootctl media: prune-profile-media to remove old avatar and header images fetch-profile-media to download missing avatar and header images basically undoin...
- Embed this notice
  Jan Wildeboer 😷:krulorange: (jwildeboer@social.wildeboer.net)'s status on Saturday, 26-Nov-2022 21:37:52 JST Jan Wildeboer 😷:krulorange:
  in reply to
  @fiftyfourpark @emanuele @mastohost @RyunoKi @aral IMHO it hardly deserves to be called a cache when it is stored indefinitely ;) And seeing how my current plan limits storage to 50 GB, it becomes next to impossible to run an instance for longer than a few years without being forced to upgrade to the next price level :(
  
  In conversation Saturday, 26-Nov-2022 21:37:52 JST permalink
- Embed this notice
  Fifty Four Park (fiftyfourpark@mastodon.54park.de)'s status on Saturday, 26-Nov-2022 21:38:34 JST Fifty Four Park
  in reply to
  @emanuele @mastohost @RyunoKi @aral wow that‘s huge, let‘s hope that clearing Avatars/Headers cache is something that will be possible in future. I keep my fingers crossed that there are people out there that see the issue and are willing and able to help with their skills.
  
  In conversation Saturday, 26-Nov-2022 21:38:34 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 23:04:30 JST Masto.host
  in reply to
  @mwfc It's not only the followers/following. If someone for example comments in your post, the avatar/header/profile info is cached, even if they nobody follows that user in that server.
  @jwildeboer @fiftyfourpark @emanuele @RyunoKi @aral
  
  In conversation Saturday, 26-Nov-2022 23:04:30 JST permalink
- Embed this notice
  …might work for coffee… (mwfc@chaos.social)'s status on Saturday, 26-Nov-2022 23:04:31 JST …might work for coffee…
  in reply to
  @jwildeboer @fiftyfourpark @emanuele @mastohost @RyunoKi @aral
  But then we are talking what? 28k Followers data in Aral's case, 5k from people Aral follows? Or is this from using the webinterface and scrolling through feeds?
  
  In conversation Saturday, 26-Nov-2022 23:04:31 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 23:05:54 JST Masto.host
  in reply to
  @Eelcom No. There is currently no command in tootctl to delete old avatars/headers or something similar. There tootctl commands for attachements and preview_cards but not avatars/headers.
  @aral @jwildeboer @fiftyfourpark @emanuele @RyunoKi
  
  In conversation Saturday, 26-Nov-2022 23:05:54 JST permalink
- Embed this notice
  Eelco Maljaars 🇳🇱 🇪🇺 (eelcom@mastodon.nl)'s status on Saturday, 26-Nov-2022 23:05:58 JST Eelco Maljaars 🇳🇱 🇪🇺
  in reply to
  @aral @jwildeboer @fiftyfourpark @emanuele @mastohost @RyunoKi aren't older/unused avatars etc not cleared out by running the media prune job with tootctl?
  
  In conversation Saturday, 26-Nov-2022 23:05:58 JST permalink
- Embed this notice
  Aral Balkan (aral@mastodon.ar.al)'s status on Saturday, 26-Nov-2022 23:06:00 JST Aral Balkan
  in reply to
  @jwildeboer Right, so basically a cache should never contain any information that cannot be recreated. You should always be able to delete a cache without losing data. If you’re losing data by clearing the cache, it’s not a cache, it’s an unreliable database :)
  @fiftyfourpark @emanuele @mastohost @RyunoKi
  
  In conversation Saturday, 26-Nov-2022 23:06:00 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Saturday, 26-Nov-2022 23:07:24 JST Masto.host
  in reply to
  @jwildeboer Only avatars and headers there are no way to remove. Preview cards are. As I do for everyone on Masto.host, described here https://masto.host/re-mastodon-media-storage/ the settings I use.
  @Eelcom @aral @fiftyfourpark @emanuele @RyunoKi
  In conversation Saturday, 26-Nov-2022 23:07:24 JST permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: masto.host
    
    RE: Mastodon media storage
    
    from Hugo Gameiro
    
    To clarify, when you or another user in your server follows a remote user, from that moment on, all new posts from that remote user start to federate with your server. By federate, you can…
- Embed this notice
  Jan Wildeboer 😷:krulorange: (jwildeboer@social.wildeboer.net)'s status on Saturday, 26-Nov-2022 23:07:31 JST Jan Wildeboer 😷:krulorange:
  in reply to
  @Eelcom @aral @fiftyfourpark @emanuele @mastohost @RyunoKi Nope. Avatars and preview panes stay indefinitely, eating up more and more storage and database space, it seems.
  
  In conversation Saturday, 26-Nov-2022 23:07:31 JST permalink
- Embed this notice
  Masto.host (mastohost@mastodon.social)'s status on Sunday, 27-Nov-2022 01:26:41 JST Masto.host
  in reply to
  
  @emanuele That's all available in `tootctl` tool provided with Mastodon
  
  In conversation Sunday, 27-Nov-2022 01:26:41 JST permalink
- Embed this notice
  Emanuele (emanuele@m.divita.eu)'s status on Sunday, 27-Nov-2022 01:26:46 JST Emanuele
  in reply to
  @mastohost @aral @RyunoKi do you have any script you could share with us?
  
  In conversation Sunday, 27-Nov-2022 01:26:46 JST permalink
- Embed this notice
  Julian Oliver (julianoliver@mastodon.social)'s status on Sunday, 27-Nov-2022 01:36:40 JST Julian Oliver
  in reply to
  @mastohost @RyunoKi @emanuele @aral Recently tootctl cleared 330GB on one of the instances I sysadmin, all content cached from federated nodes.
  
  In conversation Sunday, 27-Nov-2022 01:36:40 JST permalink

Public

Conversation

Notices

Feeds