Notices by Terence Eden’s Blog (blog@shkspr.mobi)

As you may have read, BotsIn.Space is closing down, I have lots of automated bot accounts living on the Fediverse - and I want them to continue posting. Installing and maintaining an entire Mastodon instance sounds like hard work. Paying people to host my stuff feels like putting my fate in someone else's hands.

Say… didn't I write my own ActivityPub server? Why, yes! Yes I did!

I took the code and stripped it down to the bare essentials. All you need to do is upload two files0 - index.php and .htaccess - fill in your details, and you're done.

Get the ActivityBot source code on GitLab.

There's no database, no containers, no caching. It is as simple as I could make it

This bot can do the following:

• 🔍 Be discovered on the Fediverse
• 👉 Be followed by other accounts
• 🚫 Be unfollowed by accounts
• 📩 Send messages to the Fediverse
• 🖼️ Attach an image & alt text to a message
• 🕸️ Autolink URls, hashtags, and @ mentions
• 🚚 Move followers from an old account
• 🔏 Verify cryptographic signatures
• 🪵 Log sent messages and error.

That's it! Here's what it doesn't do:

• ❌ Receive messages (other than follows and unfollows)
• ❌ Send private messages
• ❌ Thread replies
• ❌ Delete or update a post
• ❌ Create Polls
• ❌ Attach multiple images
• ❌ Set focus point for images
• ❌ Set sensitivity for images / blur
• ❌ Set "Content Warning"
• ❌ Accurate support for converting user's text to HTML
• ❌ Cannot be discovered by Lemmy instances

Grab a subdomain (don't buy a whole new domain name!) and stick this code on it. You'll have an ActivityPub bot running in minutes.

You can follow one of my bots @colours@colours.bots.edent.tel

Feedback very much welcome.

0. You can also upload a .env file for your configuration if you want. ↩︎

#ActivityPub #bot #fediverse

Is Open Graph Protocol dead?
https://shkspr.mobi/blog/2022/11/is-open-graph-protocol-dead/

Facebook Meta - like many other tech titans - has institutional Shiny Object Syndrome. It goes something like this:

1. Launch a product to great fanfare
2. Spend a few years hyping it as ✨the future✨
3. Stop answering emails and pull requests
4. If you're lucky, announce that the product is abandoned but, more likely, just forget about it.

Open Graph Protocol (OGP) is one of those products. The value-proposition is simple.

• It's hard for computers to pick out the main headline, image, and other data from a complex web page.
• Therefore, let's encourage websites to include metadata which tells our services what they should look at!

OGP works pretty well! When you share a link on Facebook, or Twitter, or Telegram - those services load the website in the background, look for OGP metadata, and display a friendly snippet.

Facebook Meta were the driving force behind OGP - and have now left it to fester.

• The website - https://ogp.me/ - still works.
• But the Facebook OGP Discussion Group is now full of spam.
• The Developer Mailing List is broken.
• The Google Documentation links to a dead Google+ page.
• And the GitHub Page has been archived.

And, that might be fine. Facebook Meta are a small company with limited resources. They can't afford to fund standards work indefinitely. And, anyway, OGP is complete, right? It has all the tags that anyone could ever possibly want. Why does it need any improving?

Well, that's not the case. We know, for example, that Twitter have created their own proprietary OGP-like meta tags. Similarly, Pinterest have their own as well. And even Google are going their own way with Rich Snippets.

This is annoying for developers. Now we have to write multiple different bits of metadata if we want our links to be supported on all platforms.

Standards work is never "finished". Developers want to add new features. Users want to interact with new forms of content.

Tomorrow someone is going to invent a way to share smells over the Internet. How does that get represented in an Open Graph Protocol compliant manner?

<meta property="twitter:olfactory" content="C₃H₆S"> or<meta property="facebook:nose" content="InChIKey/MWOOGOJBHIARFG-UHFFFAOYSA-N"> or<meta property="og:smell" content="pumpkin spice"> or...

We know from bitter experience that having several mutually incompatible ways to implement something is a nightmare for developers and provides a poor user-experience.

So we create standards bodies. They're not perfect, but a group of interested folks can do the hard work to try and satisfy oppositional stakeholders.

This is my plea to Facebook Meta. If you're no longer interested in improving OGP, OK. You do you. But hand it over to people who want to keep this going. Maybe it's the W3C, or IndieWeb, or Schema.org or someone. Hell, I'm not busy, I'll take it on.

Remember, if you love something, let it go.

https://shkspr.mobi/blog/2022/11/is-open-graph-protocol-dead/

#facebook #HTML #meta #metadata #ogp #standards #twitter

Social Media Blocking Has Always Been A Lie
https://shkspr.mobi/blog/2024/09/social-media-blocking-has-always-been-a-lie/

What does it mean to block someone on a social media site?

Way back in the mists of time, we dealt with trolls on Usenet with the almighty PLONK - PLaced On Newsgroup Killfile. It meant your newsreader never downloaded their posts. They could rant at you all day long, and you'd never hear from them. It's what we would nowadays call "Mute".

But, whether you're on Usenet or a modern social network, muting someone doesn't actually stop them replying to you. The miscreant can still see your posts, interact with them, quote them. And everyone on that service can see their abuse. Perhaps they will also join in?

Most modern social networks now have the concept of "Block". When Alice blocks Bob, it means Bob cannot see Alice's posts. The service doesn't deliver her content to him. If he goes looking, he can't find it. She is invisible to him.

Except, of course, that's a lie. If Bob logs out of his account, he can see Alice's public content. If he logs into an alternative account, he isn't blocked.

The block is a social signal backed up with mild technical restrictions.

What do I mean by that? Ordinarily, you will have no idea that you have been blocked by someone. They will simply vanish from your screens. You do not receive an alert that you've been blocked. Technical restrictions mean you won't see their posts, nor replies to them. The only way you might know is if you deliberately look for the person blocking you.

Seeing that you have been blocked is a "social signal". It lets you know that your behaviour was unwanted, or that your contributions weren't valued, or that someone just doesn't like you. For most people, that sort of chastisement probably induces a little shame or grief. For others, it is enraging.

Again, it isn't impossible for a blocked user to see content - but technical restrictions means it takes effort. And, it turns out, for all but the most obsessive abusers - a mild bit of UI friction is all that it takes for them to stop.

On a centralised social media platform, like Twitter and Facebook, your blocks are private. The only people who know you have blocked Taylor Swift are you, the platform, and T-Swizzle herself.

On decentralised social media platforms, it is more complicated.

Mastodon / ActivityPub lets you block a user. In doing so, you have to tell that user's server that you don't want them seeing your messages. That means your server knows about the block, their server know, and the user knows. But, crucially, there's nothing to stop a malicious server ignoring your wishes. While your server can mute all the interactions from them, there are only weak technological restrictions on their behaviour.

BlueSky / AT Protocol takes a different (and more worrying) approach. BlueSky tells everyone about your blocks. If Alice blocks Bob - the system lets everyone know. This means that if Bob starts replying to your posts, other clients will know to ignore his interactions with you. I've written more about the dangers of public blocklists over on BSky.

But, crucially, none of these systems actually block users. This isn't like that Black Mirror episode where people are literally blurred out from your eyeballs.

In all cases, a user can log out and see your public posts. They can sign in with an alternative account. And, in the case of decentralised social media, they can choose to ignore the technological restrictions you impose.

Social networks have a responsibility to keep their users safe. That means having enough friction to prevent casual abuse.

But blocking is only a social signal. That's all it ever has been. It is a boop on the nose with a rolled up newspaper. It is a message to tell someone that they might want to adjust their attitude.

You should block - and block often. You should feel empowered to curate an environment that is safe for you. But you should also understand the limitations of the technical controls which underpin these social signals.

https://shkspr.mobi/blog/2024/09/social-media-blocking-has-always-been-a-lie/

#ActivityPub #BlueSky #mastodon #SocialMedia #twitter

The unreasonable effectiveness of simple HTML
https://shkspr.mobi/blog/2021/01/the-unreasonable-effectiveness-of-simple-html/

I've told this story at conferences - but due to the general situation I thought I'd retell it here.

A few years ago I was doing policy research in a housing benefits office in London. They are singularly unlovely places. The walls are brightened up with posters offering helpful services for people fleeing domestic violence. The security guards on the door are cautiously indifferent to anyone walking in. The air is filled with tense conversations between partners - drowned out by the noise of screaming kids.

In the middle, a young woman sits on a hard plastic chair. She is surrounded by canvas-bags containing her worldly possessions. She doesn't look like she is in a great emotional place right now. Clutched in her hands is a games console - a PlayStation Portable. She stares at it intensely; blocking out the world with Candy Crush.

Or, at least, that's what I thought.

Walking behind her, I glance at her console and recognise the screen she's on. She's connected to the complementary WiFi and is browsing the GOV.UK pages on Housing Benefit. She's not slicing fruit; she's arming herself with knowledge.

The PSP's web browser is - charitably - pathetic. It is slow, frequently runs out of memory, and can only open 3 tabs at a time.

But the GOV.UK pages are written in simple HTML. They are designed to be lightweight and will work even on rubbish browsers. They have to. This is for everyone.

Not everyone has a big monitor, or a multi-core CPU burning through the teraflops, or a broadband connection.

The photographer Chase Jarvis coined the phrase "the best camera is the one that’s with you". He meant that having a crappy instamatic with you at an important moment is better than having the best camera in the world locked up in your car.

The same is true of web browsers. If you have a smart TV, it probably has a crappy browser.

My old car had a built-in crappy web browser.

Both are painful to use - but they work!

If your laptop and phone both got stolen - how easily could you conduct online life through the worst browser you have? If you have to file an insurance claim online - will you get sent a simple HTML form to fill in, or a DOCX which won't render?

What vital information or services are forbidden to you due to being trapped in PDFs or horrendously complicated web sites?

Are you developing public services? Or a system that people might access when they're in desperate need of help? Plain HTML works. A small bit of simple CSS will make look decent. JavaScript is probably unnecessary - but can be used to progressively enhance stuff. Add alt text to images so people paying per MB can understand what the images are for (and, you know, accessibility).

Go sit in an uncomfortable chair, in an uncomfortable location, and stare at an uncomfortably small screen with an uncomfortably outdated web browser. How easy is it to use the websites you've created?

I chatted briefly to the young woman afterwards. She'd been kicked out by her parents and her friends had given her the bus fare to the housing benefits office. She had nothing but praise for how helpful the staff had been. I asked about the PSP - a hand-me-down from an older brother - and the web browser. Her reply was "It's shit. But it worked."

I think that's all we can strive for.

Here are some stats on games consoles visiting GOV.UK

Matt Hobbs (@TheRealNooshu@hachyderm.io)

@TheRealNooshu

20/22❤️ 29💬 1♻️ 010:45 - Mon 01 February 2021

https://shkspr.mobi/blog/2021/01/the-unreasonable-effectiveness-of-simple-html/

#HTML5 #web #WeekNotes #work

Who can reply?
https://shkspr.mobi/blog/2024/06/who-can-reply/

Vague thoughts as they enter my brainbox.

The BlueSky social network has introduced "Reply Gating" - it looks like this:

You can write your hot take on Taylor Swift and not be inundated by weirdos replying to you. Nifty!

This is nothing new. Twitter has it. Facebook has the concept of "audiences" to restrict who your post is visible to.

And, of course, blogging has this! There is a comment form at the bottom of this page - and I moderate it. If you post something stupid, I don't have to subject my audience to your inanities. I can (and do) block users from commenting.

ActivityPub doesn't have this (yet). It's much more like a public mailing list. I can block or mute you - which stops me from seeing your abuse - but doesn't stop anyone else from seeing it.

Should ActivityPub have something similar? Yeah, I reckon so. I'd like to be able to say "Anyone I know want to go to the pub tonight" and only have mutuals reply. I want to prune away spam or repetitive replies. It would be helpful to have a conversation in public that other people can't interrupt.

The UI would be complex. And the social model needs a bit of work. And there are some technical challenges around syndicating which replies should be included.

But, ultimately, social media should respond to the needs of its users.

https://shkspr.mobi/blog/2024/06/who-can-reply/

#ActivityPub #mastodon #SocialMedia

Bank scammers using genuine push notifications to trick their victims
https://shkspr.mobi/blog/2024/05/bank-scammers-using-genuine-push-notifications-to-trick-their-victims/

You receive a call on your phone. The polite call centre worker on the line asks for you by name, and gives the name of your bank. They say they're calling from your bank's fraud department.

"Yeah, right!" You think. Obvious scam, isn't it? You tell the caller to do unmentionable things to a goat. They sigh.

"I can assure you I'm calling from Chase bank. I understand you're sceptical. I'll send a push notification through the app so you can see this is a genuine call."

Your phone buzzes. You tap the notification and this pops up on screen:

This is obviously a genuine caller! This is a genuine pop-up, from the genuine app, which is protected by your genuine fingerprint. You tap the "Yes" button.

Why wouldn't you? The caller knows your name and bank and they have sent you an in-app notification. Surely that can only be done by the bank. Right?

Right!

This is a genuine notification. It was sent by the bank.

You proceed to do as the fraud department asks. You give them more details. You move your money into a safe account. You're told you'll hear from them in the morning.

Congratulations. You just got played. Scammers have stolen your life savings.

This is reasonably sophisticated, and it is easy to see why people fall for it.

1. The scammer calls you up. They keep you on the phone while...
2. The scammer's accomplice calls your bank. They pretend to be you. So...
3. The bank sends you an in-app alert.
4. You confirm the alert.
5. The scammer on the phone to your bank now has control of your account.

Look closer at what that pop is actually asking you to confirm.

We need to check it is you on the phone to us.

It isn't saying "This is us calling you - it is quite the opposite!

This pop-up is a security disaster. It should say something like:

Did you call us?
If someone has called you claiming to be from us hang up now
[Yes, I am calling Chase] - [No, someone called me]

I dare say most people would fall for this. Oh, not you! You're far too clever and sceptical. You'd hang up and call the number on your card. You'd spend a terrifying 30 minute wait on hold to the fraud department, while hoping fraudsters haven't already drained your account.

But even if you were constantly packet sniffing the Internet connection on your phone, you'd see that this was a genuine pop-up from your genuine app. Would that bypass your defences? I reckon so.

Criminals are getting increasingly good at this. Banks are letting down customers by having vaguely worded security pop-up which they know their customers don't read properly.

And, yes, customers can sometimes be a little gullible. But it is hard to be constantly on the defensive.

You can read the original story from the victim on Reddit. See more comments on Mastodon.

https://shkspr.mobi/blog/2024/05/bank-scammers-using-genuine-push-notifications-to-trick-their-victims/

#bank #CyberSecurity #phishing #scam #security

HTTP Signature Infinite Loop?
https://shkspr.mobi/blog/2024/02/http-signature-infinite-loop/

I'm trying to get my head round HTTP Signatures as they're used extensively in the Fediverse.

Conceptually, they're relatively straightforward.

You send me a normal HTTP request. For example, you want to POST something to https://example.com/data

You send me these headers:

In order to verify the contents of the message, I need to do three things:

1. Check the SHA-256 hash of the message matches the content of the "Digest" header.
2. Check the timestamp is somewhat fresh.
3. Check the signature matches.

The first is simple: base64_encode( hash( "sha256", $request_body, true ) ).
The second is a matter of opinion. I might be happy to receive messages from the distant past or far in the future. For the sake of a little clock drift, let's allow 60 seconds either way.
The third gets complicated.

First, I need to get the public key published at keyId="https://your_website.biz/publicKey".

Next, I need to know which algorithm is being used to sign the headers: algorithm="rsa-sha256"

Then, I need to know which headers - and in what order - are being signed: headers="(request-target) host date digest content-type"

So I create a string using the received details which matches those headers in that specific order:

I can verify if the signature - signature="JGQ53kEoIiMWRp9By9jajVGCOCu4n7XBeiA1uY5xLcnAxL2Y1GIgU/...==" matches by:

If that's TRUE then all is well.

But can you spot the implicit problem?

How do I get your server's public key?

I just GET https://your_website.biz/publicKey - but if your server uses something like Authorised Fetch then I have to sign my request to you.

Which means your server will need to validate my signature by obtaining my public key. Which it will get by signing a request and sending it to me. Which, before I return my public key, I will need to validate your signature by obtaining your public key. Which I will get by signing a request... and so on.

This deadlock loop is documented. The usual way around it is either for the sending server to use an instance-specific signature which can be retrieved by an unsigned request, or to allow any unsigned request to access a user's public key.

I get why things happen this way - I just wish it were easier to implement!

https://shkspr.mobi/blog/2024/02/http-signature-infinite-loop/

#ActivityPub #CyberSecurity #encryption #fediverse #http

Internationalise The Fediverse
https://shkspr.mobi/blog/2024/02/internationalise-the-fediverse/

We live in the future now. It is OK to use Unicode everywhere.

It seems bizarre to me that modern Internet services sometimes "forget" that there's a world outside the Anglosphere. Some people have the temerity to speak foreign languages! And some of those languages have accents on their letters!! Even worse, some don't use English letters at all!!!

A decade ago, I was miffed that GitHub only supported some ASCII characters in its project names. There's no technical reason why your repo can't be called "ഹലോ വേൾഡ്".

Similarly, I'm frustrated that Mastodon (the largest ActivityPub service) doesn't allow Unicode usernames and has resisted efforts to change.

So I built a small ActivityPub server which publishes content from an Actor called @你好@i18n.viii.fi - it is only a demo account, but it works!

Some ActivityPub clients report that they are able to follow it and receive messages from it. Others - like Mastodon - simply can't see anything from it. Take a look at the replies on Mastodon to see which services work. You can also see some of its posts on the Fediverse.

The ActivityPub specification says:

Building an international base of users is important in a federated network.
Internationalization

I can't find anything in the specifications which limits what languages a username can be written in. But there are a few clues scattered about.

The user's @ name is defined by preferredUsername which is:

A short username which may be used to refer to the actor, with no uniqueness guarantees.
4.1 Actor objects

There's nothing in there about what scripts it can contain. However, later on, the spec says:

Properties containing natural language values, such as name, preferredUsername, or summary, make use of natural language support defined in ActivityStreams.
4. Actors

So it is expected that a preferred username could be written in multiple scripts. Which implies that the default need not be limited to A-Z0-9.

The ActivityStreams specification talks about language mapping.

Finally, the ActivityPub specification has some examples on non-Latin text in names.

So, I think that it is acceptable for usernames to be written in a variety of non-Latin scripts.

There are usually a few objections to "Unicode Everywhere" zealots like me. I'd like to forestall any arguments.

Well, what about them? ASCII has plenty of similar looking characters. I doubt most people would notice when a capital i is replaced by a lower L - and vice-versa. Similarly the kerning issue of an r and n looking like an m is well known. Are mixed language homographs more dangerous? I don't think so.

Well, what if they do? Maybe not being found by people who can't type your language is a feature, not a bug. But, anyway, clients can let users search for other people, or copy and paste their names.

It is up to a client to decide how they want to render text input. The "problems" of strange Unicode combinations are well known. This is not a hard computer-science problem.

The spec makes clear this is allowed.

I have no evidence for this. But I bet you'd get pretty frustrated if you had to switch keyboard just to type your own name, wouldn't you? In any case, why can't I have a username of @😉

If you build ActivityPub software, give some thought to the billions of people who don't have names which easily fit into ASCII.

If your software can see @你好@i18n.viii.fi and its posts, please let me know.

https://shkspr.mobi/blog/2024/02/internationalise-the-fediverse/

#ActivityPub #fediverse #i18n #mastodon #unicode

Rebuilding FourSquare for ActivityPub using OpenStreetMap
https://shkspr.mobi/blog/2024/01/rebuilding-foursquare-for-activitypub-using-openstreetmap/

I used to like the original FourSquare. The "mayor" stuff was a bit silly, and my friends never left that many reviews, but I loved being able to signal to my friends "I am at this cool museum" or "We're at this pub if you want to meet" or "Spending the day at the park".

So, is there a way to recreate that early Web 2.0 experience with open data and ActivityPub? Let's find out!

This quest is divided into two parts.

1. Get nearby "Points of Interest" (POI) from OpenStreetMap.
2. Share a location on the Fediverse.

OpenStreetMap is the Wikipedia of maps. It is a freely available resource which anyone can edit (if they're skilled enough).

It also comes with a pretty decent API for querying things. For example, nw["amenity"]({{bbox}}); finds all "amenities" near a specific location.

As you can see, it has highlighted some useful areas - a pharmacy and a pub. But it has ignored other useful locations - the train station and the park. It has also included some things that we may not want - bike parking and a taxi rank.

What API call is needed to get useful locations of of OverPass?

It's possible to specify the type of thing to find using nw["amenity"="restaurant"]; - but adding every single type of thing would quickly end up with a very large query containing hundreds of types.

It is also possible to exclude specific types of places. This retrieves all amenities except for fast food joints:

Again, that would be complex.

Perhaps one solution is just to return everything and let the user decide if they want to check in to a telephone kiosk or a fire hydrant? That's a bit user-hostile.

Instead, this query returns everything which has a name nw["name"]({{bbox}});

That cuts out any unnamed things - like park benches and car-sharing spots. But it does add named roads and train lines.

It is possible to use filters to exclude results from OverPass. The best that I can come up with is: nw["name"][!"highway"][!"railway"][!"waterway"][!"power"]({{bbox}});

That gets everything which has a name, but isn't a highway or railway or waterway or powerline. It isn't perfect - but it will do!

This is the query which will retrieve the 25 nearest things within 100 metres of a specific latitude and longitude. It includes the name and any other tags, the location, and the OSM ID.

overpass-api.de/api/interpreter?data=[out:json];nw["name"][!"highway"][!"railway"][!"waterway"][!"power"](around:100,51.5202,-0.1040);out center qt 25;

There's good news and bad news here. Firstly, ActivityStreams (which are subscribed to in ActivityPub) supports the concept of "Place" and "Location".

Once the user has a latitude and longitude, the can share it - along with a message, photo, or anything else.

Something like:

For example, here's a PixelFed post with an attached location - and this is the JSON representation. That status can be reposted into other social networks.

It is worth noting that Mastodon doesn't (natively) support location - if you view my repost of that PixelFed post you'll see there's no location metadata attached. That's OK! It just means that the status needs to include human-readable data.

Similarly, Mastodon doesn't support the arrive vocabulary. So this will be limited to a message with a location attached.

Other ActivityPub services do support location.

Well… that's a job for next week. Probably!

• Building a web site which gets the user's location is easy.
• Getting the data from OverPass should be straightforward.
• Creating an ActivityPub server which can post geotagged notes into the Fediverse might be a little beyond my skillset! Some testing with Darius Kazemi's AP Glitch suggests this should work.

If you'd like to help, please leave a comment.

https://shkspr.mobi/blog/2024/01/rebuilding-foursquare-for-activitypub-using-openstreetmap/

#ActivityPub #fediverse #FOURSQUARE #geolocation #OpenStreetMap