Notices by Ayo (ayo@lonely.town)

(Finally) got myself a UPS. Perfect opportunity to enrich the most useless graph in my home monitoring with more metrics that just repeat the same value!

"Your password must contain symbols but please don't try to hack us"

> Subject: Please reply asap

Plot twist: it was neither a scammer nor in any way urgent.

Family did not appreciate the beauty of my latest art acquisition.

The Brave search crawler does not identify itself as a crawler, but does honor robots.txt rules for Googlebot.
https://safe.search.brave.com/help/brave-search-crawler

Sadly, that's a sensible decision. Also sadly, I am now unable to whitelist them while keeping Googlebot blocked. :blobshrug:

Me: sacrifice features and convenience by enabling all anti-tracking and fingerprinting options in Librewolf.

Also me: browse the web from a static IP that uniquely identifies me.

I found (and fixed) a memory leak. :blobsmilesweat:

Oh god, it actually works.

So I've been working on a new web development framework for Perl, with a healthy dose of NIH: comes with a custom JSON implementation, PostgreSQL client, process manager and a bunch of other handy utilities. It's now running live on https://manned.org/

3800 lines of C, 1500 lines of Perl and 1800 lines of tests. Still got a bunch of features and improvements planned, but version 0.1 is live. :blobcheer:

https://code.blicky.net/yorhel/fu
https://metacpan.org/pod/FU

Yeah, this cleanup was badly needed.

Things I learned while interacting with PostgreSQL through the "binary" data format:

- Data isn't transferred in the same format as stored on disk or used internally in Postgres, there's type-specific conversions going on. I assume to offer improved portability and stability.

- Even in the binary format, textual data is converted between the database encoding and whatever you set as 'client_encoding'.

- jsonb columns are converted to/from textual JSON; the 'b' part stays internal to Postgres.

- Dealing with the more complex types (like arrays) is not as painful as I had anticipated; the binary format might actually be easier to work with than the generally recommended textual formats.

- Big-endian everywhere. :blobcatnotlikethis:

- Judging from the Postgres code I've skimmed through, I don't expect the binary format to have much of a performance advantage compared to text, but I still have benchmarks to perform.

I was wondering why Firefox felt extremely sluggish today.

Turns out I had accidentally put my laptop in the fixed-400MHz-slow-mode I sometimes used to measure responsiveness of VNDB on slow devices. :blobsmilesweat:

(VNDB is still usable, most other sites not so much...)

Against better judgement, I spent the past week writing my own JSON encoder and decoder for Perl. :blobshrug:

I'm happy with the result though. It's 3x smaller than JSON::XS, 7x smaller than Cpanel::JSON::XS, yet it has all the features I need and compares favorable in terms of performance, too. Was a good opportunity to learn more about perl internals as well. :blobcatjustright:

Looking into Raku again and, while the language seems pretty neat, the bare-bones standard library really throws a wrench in the fun. Gentoo doesn't even package the 'zef' package manager, so I'd have to grab that from source manually. Even with that done, there's the typical wild west of uncoordinated package development: always a bunch of competing packages for the same functionality, many unmaintained, some in eternal beta (and not promising a stable API), some missing crucial functionality, and some depending on 20+ other packages. Choose your poison.

Meanwhile in Crystal land I've written several useful programs that had no dependencies besides the compiler and standard library, both of which are within reach of a single `emerge`.

The 'zef' package manager itself puts it best:

"rakudo must be able to parse json, so it doesn't make sense to require a dependency to parse it"
https://github.com/ugexe/zef/blob/main/lib/Zef.rakumod#L5-L6

I wondered how zef fetches packages when Raku doesn't come with a http(s) client... it shells out to curl or wget. Yeah, that's the kind of solution you end up with. :blobshrug:

@corbet @johnefrancis @LWN
Struggling with likely the same bots over here. I deployed a similar tarpit* on a large-ish site a few days ago - taking care not to trap the good bots - but can't say it's been very successful. It might have taken some load off of the main site, but not nearly enough to make a difference.

One more thing I'm considering is prefixing all internal links with a '/botcheck/' path for potentially suspicious visitors, set a cookie on that page and strip that prefix with JS. If the cookie is set on the /botcheck/ endpoint, redirect to the proper page, otherwise tarpit them. This way the site would still work as long as the user has *either* JS or cookies enabled. Still not perfect, but slightly less invasive than most common active defenses.

*) https://code.blicky.net/yorhel/infinite-slop

> 28 files changed, 627 insertions(+), 1102 deletions(-)

Getting rid of old Elm code from VNDB, one form at a time. :blobcathappy:

@lanodan I threw some VNDB data at the templates for that instance. :blobcathappy:

My latest silly project: Infinite Slop, a template-based random page generator to waste resources of stupid crawlers.

https://code.blicky.net/yorhel/infinite-slop

Now to find some fun ways to direct bots to it.

I had the brilliant idea to embed back-end debug data to pages as links:

<a href="/debugjson?c={..json here}">debug info</a>

Which the '/debuginfo' enpoint would then nicely format.

...but that doesn't seem to work when the debug info easily exceeds 32 KiB, even after trying some compression. Firefox refuses to even follow the link. :blobcatthinking: