Notices by piefedadmin (piefedadmin@join.piefed.social)

Fediverse traffic is pretty bursty and sometimes there will be a large backlog of Activities to send to your server, each of which involves a POST. This can hammer your instance and overwhelm the backend’s ability to keep up. Nginx provides a rate-limiting function which can accept POSTs at full speed and proxy them slowly through to your backend at whatever rate you specify.

For example, PieFed has a backend which listens on port 5000. Nginx listens on port 443 for POSTs from outside and sends them through to port 5000:

To this basic config we need to add rate limiting, using the ‘limit_req_zone’ directive. Google that for further details.

This will use up to 100 MB of RAM as a buffer and limit POSTs to 10 per second, per IP address. Adjust as needed. If the sender is using multiple IP addresses the rate limit will not be as effective. Put this directive outside your server {} block.

Then after our first location / {} block, add a second one that is a copy of the first except with one additional line (and change it to apply to location /inbox or whatever the inbox URL is for your instance):

300 is the maximum number of POSTs it will have in the queue. You can use limit_req_dry_run to test the rate limiting without actually doing any limiting – watch the nginx logs for messages while doing a dry run.

It’s been a while since I set this up so please let me know if I mixed anything crucial out or said something misleading.

https://join.piefed.social/2024/04/17/handling-large-bursts-of-post-requests-to-your-activitypub-inbox-using-a-buffer-in-nginx/

#nginx #webPerformance

By default, all posts show up in search results on #PieFed, #Lemmy and #Kbin. But in a first for the threadverse, PieFed has just added some privacy features that Mastodon had for a long time – being searchable is now optional!

Un-tick the “My posts appear in search results” checkbox in your settings and not only will your posts be hidden from the PieFed search on your instance, but on all other PieFed instances too (yes, it federates, but only to PieFed instances).

But wait, there’s more. Google will not add your profile or any of your posts to it’s index (because of the <meta name=”robots” content=”noindex”> tag used when rendering your posts) AND comments you made on other people’s posts won’t be indexed by Google either (because of the <!–googleoff: all–> tag).

So if you’re tired of living in a fishbowl, want a bit of privacy or would rather people can’t follow you around, PieFed is here for you.

https://join.piefed.social/2024/03/12/piefed-privacy-control-your-search-visibility/

#fediverse #Kbin #Lemmy #piefed #privacy #search #threadverse

Recently @siderea wrote a fantastic thread about social homogeneity, moderation, the design of social platforms and what they could be. They covered a lot of ground and I can’t respond to it all so I’ll just pick some highlights

I cannot tell you how many conversations I have seen about the topic of “moderation” and how necessary it is in which nobody has ever bothered to set down what exactly it is that they think a moderator is supposed to accomplish.

I mean, it’s all of them. I’ve been on the internet since the 1980s, and I have never seen anyone stop and actually talk about what they thought moderators were trying to do or should try to do.

That sounds easy. I’ll take a shot at that, below.

Also they draw a parallel between designing buildings and designing social platforms:

Why should our societies tolerate the existence of *irresponsibly* designed and operated social media platforms, that increase violence and other antisocial behavior?

Primarily buildings are built to be used, and as such they are tools, and we judge them, as we do all tools, by how fit they are for their purpose, whatever that might be.

And the purposes of buildings are to afford various ways of people interacting or avoiding interacting.

So architects think a lot about that. It’s a whole thing.

Those who put together social media platforms need to think about the same sort of thing.

Preach!

The upshot is that we can do better than what we have in the past. We can go beyond the bare minimum of “delete the spam, ban the nazis” moderation. When we build social software the features it has will determine what kind of moderation is possible, what kind of interactions people will have. We should be intentional about that.

I’d like to share some of my ideas for how we can do that but first, let’s get the basics covered:

Obviously every online space is different and has it’s own values and priorities. What follows is what I consider to be the minimum necessary to avoid devolving into 4chan as soon as the normies arrive.

The goal of moderators is to create a positive, inclusive, and constructive online community where users feel comfortable engaging in discussions and sharing their thoughts and ideas. To that end, their responsibilities include:

1. Enforcing Community Guidelines:
   • Moderators ensure that users adhere to the forum’s rules and guidelines. This may involve removing or editing content that violates these rules.
2. Fostering a Positive Atmosphere:
   • They work to create a welcoming and friendly atmosphere within the forum. This includes encouraging respectful communication and discouraging any form of harassment or bullying.
3. Managing Conflict:
   • Moderators intervene when conflicts arise between users, helping to de-escalate situations and resolve disputes. This may involve mediating discussions or issuing warnings to users.
4. Preventing Spam and Irrelevant Content:
   • They monitor the forum for spam, irrelevant content, or any form of disruptive behaviour. This helps maintain the quality of discussions and keeps the forum focused on its intended topics.
5. Addressing Technical Issues:
   • Moderators often assist users with technical issues related to the forum platform. This includes addressing bugs, helping users navigate the site, and forwarding technical problems to the appropriate channels.
6. Encouraging Positive Contributions:
   • Moderators actively encourage users to contribute positively to discussions. This can involve highlighting valuable contributions, providing constructive feedback, and recognizing members for their positive engagement.
7. Applying Consequences:
   • When necessary, moderators may apply consequences for rule violations, such as issuing warnings, temporary suspensions, or permanent bans. This ensures accountability and helps maintain a healthy community.
8. Staying Informed:
   • Moderators stay informed about the forum’s community and culture, as well as any changes in policies or guidelines. This helps them address issues effectively and stay responsive to the evolving needs of the community.
9. Collaborating with Community Members:
   • Moderators listen to concerns and feedback from the community. Taking a collaborative approach helps build trust and ensures that the moderation team understands the community’s needs.

Ok, cool. But:

When we think about moderation tools for a platform that serves millions of people, we are shaping the nature of social interactions on a grand scale. As we engineer these virtual societies, the question we need to ask ourselves is, “What is the nature of the society we want to create?” and within that, “What do we want moderation to accomplish that supports that nature?” and eventually “What software features do moderators need to do their work?”

We want to create an ideal society where everyone is safe, respected, empowered, entertained and encouraged to grow and find meaning according to their individual free choices. Members of this online society contribute meaningfully and positively to the rest of society, support the actualisation of human rights for all and work to help democracy to live up to it’s promise.

Remember the 1990s, when the internet hadn’t been corrupted yet? Yeah. I do.

Defining the Role of Moderation

Moderation should not be a passive, reactive role. Instead, it should be proactive, shaping the community’s social dynamics intentionally. The first step towards this is defining what our platforms aim to achieve. Do we want a space for free and open discussions, a supportive community, or a platform for specific interests? This vision will shape the guidelines we develop, the tools we use, and the strategies we implement.

Developing Clear Guidelines and Empowering Moderators

Once we have our vision, we need to create a set of rules that align with this vision. These guidelines should be clear, easily accessible, and comprehensive. Moreover, we need to empower our moderators with the right tools and authority to enforce these guidelines. This can include features for deleting posts, banning users, or moving discussions.

Investing in Technology

Incorporating technology is crucial in supporting our moderators. Automated moderation tools can detect and remove inappropriate content, while algorithms can promote high-quality posts. Technology can also help in combating challenges like trolls who use new IP addresses to create accounts. Techniques like browser fingerprinting can identify users regardless of their IP, and restrictions on new accounts can deter trolls.

Addressing Complex Issues

Online communities also need to grapple with complex issues such as the formation of high-control groups, disinformation propagation, social isolation, and internet addiction. Tackling these problems requires more advanced tools and strategies:

• For high-control groups, we need to implement robust reporting systems and use AI tools to detect patterns of manipulation.
• To combat disinformation, we need to establish strong fact-checking protocols, possibly collaborating with external fact-checking organizations.
• To mitigate social isolation and internet addiction, platforms can implement features to promote healthier usage, like reminders to take breaks or limits on usage time.
• To manage trolls, we can use advanced techniques that track users beyond their IP address and limit the activities of new accounts until they show they can be trusted.

Continuous Evaluation and User Education

Finally, moderation should be an ongoing process of improvement and adaptation. We need to regularly review and update our strategies based on their effectiveness and changing conditions. Additionally, we need to educate our users about these issues and how to report them. An informed user base can greatly aid in maintaining a healthy community.

In conclusion, moderation in online communities is not just about maintaining order but about intentionally shaping the dynamics of these spaces. As we navigate the digital age, we must recognize the power and responsibility we hold in engineering these virtual societies, and use it to create healthier, safer, and more inclusive communities.

https://join.piefed.social/2024/03/07/moderation-the-design-of-social-platforms/

#culture #moderation #society

For a very small instance with only a couple of concurrent users a CDN might not make much difference. But if you take a look at your web server logs you’ll quickly notice that every post / like / vote triggers a storm of requests from other instances to yours, looking up lots of different things. It’s easy to imagine how quickly this would overwhelm an instance once it gets even a little busy.

One of the first web performance tools people reach for is to use a CDN, like Cloudflare. But how much difference will it make? In this video I show you my web server logs before and after and compare them.

The short answer is – before CDN: 720 requests. After CDN: 100 requests.

Usually just turning on a CDN with default settings will not help very much, you’ll need to configure some caching rules or settings. By watching your server logs for a while you’ll get a sense for what needs to be cached but check out mine for a starting point:

All these are frequently requested on my instance. Depending on the fediverse platform you have installed, you’ll probably see different patterns and so need different caching settings.

Beware of caching by URI Path because often fediverse software will return different data depending on the Accept header that the requester sets. For example, on PieFed and Lemmy instances a request by a web browser to /post/123 will return HTML to show the post to someone. But when that same URL is requested with the Accept: application/ld+json header set, the response will be an ActivityPub representation of the post! You don’t want people getting activitypub data in their browser and you don’t want to be serving HTML to other instances. Once you spot a URL you want to cache, use a tool like Postman to set the Accept header and make a fake ActivityPub request to your instance and see if you get back HTML or JSON.

Another problem that can happen is that often a response will vary depending on whether the viewer is logged in, or who is logged in. If you can figure out how to configure the CDN to pay attention to cookies or whatever headers are used for Authentication by your platform then you might be able to cache things like /post/*… I couldn’t.

The things I’ve chosen to cache by URI Path above are ones that I know don’t vary by HTTP header or by authentication.

Although we can’t use URI Path a lot of the time, we can cache ActivityPub requests by detecting the Accept: allocation/ld+json header:

This will cache all ActivityPub requests, regardless of URL. People browsing the same URLs as those used by ActivityPub will be unaffected as their requests won’t have the special HTTP header. I used a short TTL to avoid serving stale data when someone quickly edits a post straight after creating it.

There seems to be a deep vein of optimization here which I’ve only just started to dig into. These changes have made a huge difference already and for now my instance is under very little load so I’ll leave things as they are. I look forward to learning more about this in future.

https://join.piefed.social/2024/02/20/how-much-difference-does-a-cdn-make-to-a-fediverse-instance/

#cdn #cloudflare #fediverse #piefed #threadverse #webPerformance

Google provides a tool called PageSpeed Insights which gives a website some metrics to assess how well it is put together and how fast it loads. There are a lot of technical details but in general green scores are good, orange not great and red is bad.

I tried to ensure the tests were similar for each platform by choosing a page that shows a list of posts, like https://mastodon.social/explore.

PieFed and kbin do very well. pixelfed is pretty good, especially considering the image-heavy nature of the content.

The rest don’t seem to have prioritized performance or chose a software architecture that cannot be made to perform well on these metrics. It will be very interesting to see how that affects the cost of running large instances and the longevity of the platforms. Time will tell.

https://join.piefed.social/2024/02/13/technical-performance-of-each-fediverse-platform/

#akkoma #Kbin #Lemmy #mastodon #misskey #peertube #piefed #pixelfed #pleroma #webPerformance

Us sitting here with our fiber internet and recent model phones have it pretty good. But the “i” in iPhone stands for “inequality”. Most people in the world still have pretty bad internet and old/slow phones. For a platform to be widely adopted and to serve the needs of those who often miss out, it needs to be frugal in network and cpu usage.

Due to Lemmy’s javascript-heavy software architecture, visiting a Lemmy home page involves downloading a lot of JavaScript – about 4.0 MB of it. And this only gets you 20 posts! Also community thumbnails, even if displayed as a 22px by 22px icon are served directly from their home instances, unresized, which can often be multiple megabytes in size. The home page of lemmy.nz is currently weighing over 9 MB.

Kbin’s home page comes in at a respectable 1.65 MB due to relying less on JavaScript. However it is let down by not using loading=”lazy” on images so they all need to be loaded immediately and by generating post thumbnails that are twice as big as they need to be.

The PieFed home page, showing 5x more posts than Lemmy, weighs between 700 and 930 KB, depending on which posts are shown. In low bandwidth mode, the home page is only 320 KB due to not having any thumbnails.

When viewing a post, we can assume various assets (CSS, JS and some images) are cached due to loading the home page first.

The picture looks similar when viewing a post, which is a bit surprising. One of the usual benefits of the JS-heavy SPA architecture used by Lemmy is that once all the ‘app’ is loaded into the browser, subsequent pages only involve a small API call. However, going to a page in Lemmy involves two API calls (one for the page and one for the comments) both of which return quite a bit of data. If you look at the ‘get the comments on this post’ JSON response you can see the developers have fallen into the classic SPA pitfall of “over-fetching“. They’re retrieving a whole haystack from the backend and then using JavaScript to find the needle they want, which involves transferring the haystack over the internet. Ideally the backend would find the needle and just send that to the frontend.

Kbin sends more data than it needs to when viewing a post, again because of not using loading=”lazy” which causes every profile picture of the commenters to be loaded at once. Making this simple fix would bring the weight down, from ~800 KB to around 50 KB.

PieFed only sends 10 KB – 30 KB to show a post, but it varies depending on the number and length of comments. This could be reduced even more by minifying the HTML response but with PieFed under active development I prefer the source to be as readable as possible to aid in debugging.

This is no accident. It is the result of choices made very early on in the development process, well before any code was written. These choices were made based on certain priorities and values which will continue to shape PieFed in the future as it grows. In a world where digital access remains unequal, prioritizing accessible and fast-loading websites isn’t just about technology; it’s a step towards a more inclusive and equitable society.

https://join.piefed.social/2024/02/09/comparing-network-utilization-of-lemmy-kbin-and-piefed/

#Kbin #Lemmy #piefed #threadverse #webPerformance

As of today, PieFed includes a ‘theme engine’ which makes it easier for people with low or no Python skills to change how PieFed looks and behaves.

Let’s start with the app/templates/themes directory in the PieFed codebase. In this directory there will be a sub-directory for each theme that is available. As of this writing there is only one, called ‘high_contrast’. If you want to make a new theme, copy the high_contrast directory and rename the copy to something 20 characters or less. Let’s pretend your theme is called ‘my_theme’.

Inside ‘high_contrast’ is high_contrast.json. This file contains a bit of information about the theme, to improve the user experience of choosing which theme to enable. Currently the only information in there is a user-friendly name for our theme but I might add more in future. In our new theme we to rename high_contrast.json to be the same as the directory containing it, or ‘my_theme.json’ in this case. Also change the user friendly name in the JSON to something like “My Theme”. Take care not to change the double quotes.

Also in the theme directory are two files: styles.css and scripts.js. You can remove the high_contrast CSS and put your own. high_contrast’s scripts.js is empty and is just a placeholder to encourage you to add your own JavaScript.

In PieFed on the main menu, go to Account -> Settings and check if your new theme is shown as an option in the Theme field. If not then you messed up the naming of the theme directory or theme.json file. The theme setting here will change the theme you use while browsing PieFed – to change it for everyone go to Admin -> Misc Settings and set the ‘Default theme’ option.

Editing styles.css and scripts.js will probably be enough for most people. But in addition to that you can copy any .html file from app/templates into your theme and that copy will be used instead of the default one. For example, the home page template is app/templates/index.html so if you copied it to app/templates/themes/my_theme/index.html then any edits you make to the copy will be used to render the home page in a different way than the stock PieFed theme would!

If you want to theme a template in a sub-directory, like app/templates/post/post.html then the correct place for your copy will be app/templates/themes/my_theme/post/post.html.

The .html templates are a mixture of HTML and a language similar to Python called Jinja2.

I very much look forward to seeing what people come up with!

PieFed is licensed under the open source AGPL license and themes are considered derivative works. As a result, themes need to be licensed under the AGPL as well.

This means that anyone who distributes or shares their PieFed themes must do so under the terms of the AGPL, ensuring that the source code of the theme remains open and accessible to others. If you do not distribute or share the theme with others then you do not need to share the source code with the public.

Using the theme to serve a publicly-accessible PieFed instance is “distribution” so you need to be able to make the theme source code available to everyone, either as a downloadable file or a git repository. If you choose to send PieFed a PR of your theme it will be gratefully accepted.

https://join.piefed.social/2024/02/07/changing-piefeds-appearance-with-themes/

#jinja2 #piefed #python #threadverse

Here it is 🙂 https://join.piefed.social/2024/01/22/an-introduction-to-the-piefed-codebase/

There are quite a few communities that are entirely image posts so presenting them as a vertical list of thumbnails doesn’t really make the best of what is there.

To improve on that I’ve made use of the common web design pattern called ‘masonry’, where the images are arraigned like bricks in a wall. Check it out:

https://piefed.social/c/aww@lemmy.world

https://piefed.social/c/artporn@lemm.ee (wide tile – best on large monitors)

On each tile there is the title of the post at the bottom which can be clicked on to view the post and it’s comments. This footer area could be improved with voting buttons and perhaps the number of comments, in future.

This doesn’t work as well for meme communities as memes often contain a lot of text, which gets squished. Perhaps there needs to be a 2 or 3 column version with larger thumbnails.

https://join.piefed.social/2024/01/21/optimizing-image-communities-introducing-grid-view-masonry-for-improved-image-display/

#fediverse #masonry #piefed

The last 5 days have been very busy, as expected. Things always get intense for a while when other people start using new software out there in the real world.

The main purpose of this phase is to fix problems that show up and to get enough #ActivityPub data flowing through the system to highlight any fundamental design problems with either the structure of the database or the code. I have been pleasantly surprised at how well it handles a high volume of incoming Activity data. After 5 days of operations there are now 7,000 posts, 28,000 comments and 500,000 votes while all database queries are still returning their results in under 100ms and CPU load average is under 1.0 (under 0.20 during quiet times). It seems to be about 33% as heavy as my #Kbin instance (running on the same server) was.

As well as fixing bugs and monitoring performance, I’ve added a slew of improvements and features:

• Comments are ranked by hotness, not just ‘top’ as they were at the start.
• Image posts can be viewed ‘inline’, without having to go to the post itself. Look for the magnifying-glass icon. This will the body of text posts in the same way, soon.
• Community subscriptions can be imported from #Lemmy
• Bell icon on community heading to be notified about new posts in the community. Notifications about replies to to your post or comment can be toggled using a similar bell icon on your post/comment.
• Low bandwidth mode – no images and less Javascript
• A notice on Beehaw posts to be mindful of their higher quality standards
• Topics in breadcrumb – better community discoverability
• PieFed groups to join

Coming up soon, no promises when tho:

• Send an email about unread notifications (with unsubscribe, naturally)
• Federation fixes, especially issues with Communities hosted on PieFed not sending posts out to other servers.
• Masonry-based layout for image-heavy communities like this one.
• Themes, including dark ones.
• Code introduction and development environment setup screen-casts, for future developers.

https://join.piefed.social/2024/01/10/beta-test-update/

#activitypub #fediverse #Kbin #Lemmy #piefed #python