Notices by Siderea, Sibylla Bostoniensis (siderea@universeodon.com), page 2

[break over, resuming]

Now, I certainly don't have a proposed one right answer to what a social media platform should be doing to solve all of these ensuing problems, and I certainly hope nobody thought I did.

But what I do have to propose is a set of attitudes and approaches to building out a social media platform to try to avoid some of the bad outcomes that other platforms have experienced.

My biggest point here is to simply not have a kind of foolish hubris of thinking that because something hasn't been a problem *so far*, that it's been solved.

As with so many things, I think it helps enormously to look into the history of previous attempts to get advanced warning of the circumstances one may find oneself in. And, of course in the case of social media, by "may" I mean "almost certainly will".

There are things that most definitely do not need to be surprises anymore.

🧵

A couple decades ago, Clay Shirky gave a talk which he then published as an essay, "The Group Is Its Own Worst Enemy", about how over and over and over again people who develop online social spaces get surprised by things that happened on their online space – thing which had happened previously on OTHER parties' online social spaces, and which those social spaces' governance parties had attempted to warn others about.

Now, I have a bunch of reservations about specific details in that essay, but he was sure right about how over and over and over again Bad Things happen to social platforms, and the governance parties who lived through them try to warn others, and they're pretty reliably ignored.

Maybe we could not do that this time?

🧵

Of course architects can choose to build buildings to meet other criteria, besides the effects on the people that interact with them. They can choose to make buildings that support the environment, or save the owners' money, or achieve some political end. They can also build buildings to have social effects not just through their affordances but through aesthetics, such as being beautiful to improve a neighborhood's appearance or to aggrandize an aristocracy.

But primarily buildings are built to be used, and as such they are tools, and we judge them, as we do all tools, by how fit they are for their purpose, whatever that might be.

And the purposes of buildings are to afford various ways of people interacting or avoiding interacting.

So architects think a lot about that. It's a whole thing.

Those who put together social media platforms need to think about the same sort of thing.

🧵

And I want to point out something else that's probably crucial to learning from past mistakes.

When we build a social media platform – when we build anything to allow people to interact in the internet – we are doing something very like building a planned city. We are making decisions about the structures through which people will flow and move and rest and encounter one another and interact with one another.

When architects are designing physical buildings and when urban planners are laying out physical cities, they make decisions about physical structures with the intention of those structures shaping human behavior. People who build amphitheaters are people who want there to be public addresses that many people here, whether political speech or entertaining theater. People who build temples are people who want there to be collective religious worship. People who build roads want there to be travel.

🧵

It's unfortunate that the term "social engineering" has a previous meaning as a slang term among computer programmers for a kind of attack on a system that leverages human frailty as opposed to faults in the software, because this – the design of social media platforms – is truly *social engineering*.

From where I sit, with a foot in both the technological and the social sciences, it seems really clear to me that there is no general sense that there is such a field as the engineering of online society. Not their underlying technologies, but the use of technological deployment to instantiate social spaces, that bring about certain social realities.

This is not a thing that is taken seriously. To the contrary, it's treated quite lightly.

🧵

We need to be very conscious that the decisions that are made of how a platform works are decisions that affect how the people who use that platform will interact.

There should be a kind of intentionality – which is something I think Mastodon is doing way better at than a lot of social media projects – around functionality decisions.

But that intentionality has to go beyond merely meaning well. Good intentions poorly informed result in bad outcomes that were never intended but are, nevertheless, still bad.

There is a lot to be said for realizing that decisions for how social media platforms *work* are deliberate attempts to shape – to *engineer* really – human social life on a huge scale. On a scale so huge in fact, that it is not wrong to describe it as trying to *engineer societies*.

🧵

Engineers tackle well-specified problems, and if the problems they are asked to tackle are not well-specified, they'll either nope out or they'll come up with their own spec.

It would probably do us good to spec out problems we think we're solving more precisely.

I cannot tell you how many conversations I have seen about the topic of "moderation" and how necessary it is in which nobody has ever bothered to set down what exactly it is that they think a moderator is supposed to accomplish.

I mean, it's all of them. I've been on the internet since the 1980s, and I have never seen anyone stop and actually talk about what they thought moderators were trying to do or should try to do.

That makes it a little tricky to evaluate whether or not moderators are given adequate tools to do their jobs. What with not actually having any agreement or understanding or even specification of what those jobs are.

🧵

The social media world is filled with people just pulling ideas out of their asses and hoping it all works out.

Folks who have been around the block a few times in a governance role have started amassing a body of lore. Case studies, observations they made in the trenches.

At the very least, availing oneself of what they have to share is a good first step.

But if we were to take this seriously as engineering, well, that suggests a few things, doesn't it?

It suggests we get a little bit more sciency about this. It suggests we start imposing a little bit of rigor.

🧵

This raises some very fundamental and quite interesting questions about what the role of moderation is on a social media platform. Is it the job of a social media platform to prevent people from using it to collaborate to commit crimes?

Historically, a lot of people who have put together social media platforms have insisted it is absolutely not the job of the platform – or the people who run it – to do that.

But if it's not the job of the platform to do that, whose job is it, when a platform, by its affordances, makes real world crimes – horrendous, very serious "real-world" crimes like actual genocide – not just more likely, but so much more likely they are effectively enabling a crime that wouldn't otherwise happen?

Why should our societies – our larger, meat-world societies – tolerate the building and operating of social media platforms that destabilize them and are detrimental to them?

🧵

This specific example is on my mind in part because of reading @kissane's article on Facebook's role in the genocide of the Rohingya in Myanmar. One of the things it mentions is that Facebook's internal apparatus for what we might call moderation was its "bullying-focused 'Compassion Team'". Like many social media platforms constructed by the sorts of people who construct social media platforms, Facebook construed the problem of moderation being one of preventing or discouraging interpersonal conflict on the platform.

But the problem unfolding in the Burmese-language parts of Facebook was not people disagreeing with one another or expressing conflict with one another. It was their *agreeing* with one another.

Agreeing to go kill their neighbors.

This was not something that was even on Facebook's radar, apparently.

🧵

Now, I'm not party to what's happening inside Reddit. I don't know the logic of their decisions. But I do know a whole lot of very thoughtful Reddit users who have spaces they moderate on Reddit have explained in great detail and length ("Concision is not our brand." - a mod from r/AskHistorians explaining on Twitter about this very thing) what their needs are and why they were objecting to Reddit turning off the API.

Reddit corporately decided that supporting those affordances was unimportant, or at least less important than something else that conflicted with them.

Reddit made a design decision that changed the nature of what moderation *could* mean on Reddit. They reduced its scope. That, in turn, changed how moderators could interact with the users they moderated, and that in turn changed how users interacted with one another.

🧵

Consider what has unfolded recently with Reddit turning off its API, such that tools its moderators relied on are no longer available to them. Reddit's structure is that it allows anyone to start their own forum and gives them authority to moderate it however – to a first approximation – they see fit. But it doesn't provide the tools necessary – nor, any longer, allow third parties to provide those tools – such that many moderator functions can be performed, so there's a limit to what kinds of moderation can happen there, and how well it can be acquitted. This has literally changed what kinds of conversations and what kinds of forums can happen on Reddit.

🧵

Or put another way, why should our societies tolerate the existence of *irresponsibly* designed and operated social media platforms, that increase violence and other antisocial behavior?

So it turns out the failure of internet culture to actually have a discourse around what even moderators are supposed to be doing is a literally lethal mistake.

And this example is merely one wrinkle in the much, much larger conversation about what moderation is, and the diversity of things that it can be, and maybe should be.

A conversation that has to happen before you can have the conversation that goes, "Okay, of the things that moderation can be, which things do we think it needs to be on our platform, and what do we need to do, in the design of our platform, to bring it into existence and make it work the way we think it should?"

🧵

It's probably pretty easy for Reddit executives to sniff and say, "Well you know the user base, they're making a big to-do about nothing; they'll figure out how to moderate without those tools, they're hardly critical."

Mastodonians are smarter than to do that, but we have a bit of a problem of falling down at the next step. It's great that people here don't scorn the idea that affordances matter to user behavior, but the next step is to actually find out how affordances actually do affect user behavior.

Like, we could imagine a more enlightened Reddit not cutting off its moderators at the knee by shutting down the API access they needed. But we could also imagine an even more enlightened Reddit than that, one that built its own versions of those moderator tools right into its own platform, so that moderators didn't have to use third party tools across the API.

But we could also imagine an even more enlightened Reddit than *that*.

🧵

Like I said, I'm not party to Reddit's internal corporate thinking. But I think it's a pretty good educated guess to say: Reddit's decision was not based upon what would optimize Reddit's social functions. When Reddit made this decision, I'm feeling pretty confident it was not a *social engineering* decision. It was not made to make Reddit function better in some social sense. Nobody made this decision thinking, "Actually, reducing the capacity of moderators to do tasks that are part of moderation will actually improve the social reality of Reddit in this particular way."

At very best, this decision was made to optimize something else in full awareness, "Yes, this will be detrimental to Reddit's social world, but it can't be helped, because of other considerations that outrank quality of social engineering right now."

But of course, the social effect on Reddit might have been simply dismissed, or discounted.

🧵

We could imagine an ultra enlightened Reddit that actually has opinions about how it wants its social world to function, and actually makes decisions like, "We don't like some things that we think are maybe a product of moderators not providing 24/7 coverage of high volume groups, so we're going to find out if there's some way, or ways, plural, of solving that problem. We'll investigate whether there are things we can do to either facilitate moderators providing 24/7 coverage, or obviating the need for moderators to provide 24/7 coverage, and then we'll evaluate whether or not it worked to remedy the things that we saw as problems that we think are being caused by that."

In the social services field, this crucial last bit is called "program evaluation".

🧵

One of the things you might notice from the fact that now twice I've mentioned there's other fields that have technical terminology that applies here: Oh hey. There are other fields that have clues that pertain. They have methodologies and other cool toys you might want to play with.

Engineering is, to a first approximation, applied science. So if you want to engineer socials, you might want to start hitting up the social scientists and people in other fields that apply social science.

This is something else Mastodon has going for it: it's got social scientists around here somewhere.

Now, I appreciate what I propose here has a bootstrapping problem. I don't know whether any of the decision makers about code, protocols, or individual instances have the capacity to enlist the help of the people on Mastodon with those professional clues, and I'm not sure Mastodon has the affordances to bring them together.

🧵

Part of what makes it so rigorous is how formal it is, with that business of deciding in advance what the criteria of success will be. That in turn requires a certain amount of serious thinking about social phenomena, and actually getting explicit about things that, frankly, usually just get hand waved through when we're talking about social media platforms. Questions like "what even do we expect our moderators to be achieving?"

It means doing hard things like asking, "Okay, we want people to 'feel more safe': how will we be able to tell that people are feeling more (or less) 'safe'? If people were feeling more or less safe, how would we know? How would we be able to measure it? To observe it in the data? What is it that we are assuming will change in people's behavior based on how safe they feel?"

In the social sciences, this is called "operationalizing" an abstraction or concept or feeling.

🧵

I think of it as sort of like calling one's shots in billiards. You decide what change you would like to see in the system you are designing for, you come up with an "intervention" (based on studying the problem, reading into other people's approaches to trying to solve the problem, and maybe doing a bunch of rounds of iterative experimentation), you decide, up front, how you will determine whether or not the problem has been solved, and then you implement the intervention, and then you check those criteria you previously identified as the ones that will determine whether or not the problem has been solved.

This is what I mean by rigor. This is pretty sciencey, no? It's not necessarily a controlled experiment, but it does have the form of an experiment. But it's not a *mere* experiment, either. It's not just a trial to see whether or not something will work. It's an attempt to actually do something that will work. With some slightly more rigorous testing as to whether it did.

🧵

And nobody goes to get these resources, nobody seeks out this expertise, who does not move past the naive arrogance of assuming there's nothing that they need to learn, that they have this social media platform thing all figured out already.

What I am hoping to achieve by this thread is to tantalize you with the evidence that there are things to know which you probably don't yet know, but would, if you only knew, like very much to know. Things that would benefit you to know.

I am hoping to enlist your curiosity in tackling to the floor the assumption you might harbor in your breast that this social media thing really isn't all that hard, you just do it in the right way, and which way is the right one is really obvious.

I am attempting to entice you with the knowledge that is out there (and insofar as there are experts in these things here on Mastodon, in here with us) into wanting that knowledge enough to go looking for it.

And to ask for it.

🧵