Notices by Fediverse Test Suite (feditest@mastodon.social)

FediTest v0.4, the test framework and test suite for the #Fediverse, was released yesterday. Lots of new stuff, check it out!

https://feditest.org/release-notes/0.4/

FediTest version 0.3 has been released!

Release notes: https://feditest.org/release-notes/0.3/

This release is mainly a maturity improvement over our first release.

/cc @activitypubtestsuite

To all you #developers implementing #SSRF protections in your #fediverse applications...

We are all in favor of those protections. But!

Have a setting that lets projects like #FediTest override it. Otherwise how can anybody test interop on anything other than on the public internet?

Mastodon has a ALLOWED_PRIVATE_ADDRESSES setting, which is one way of doing it. Or just have a setting with a default value of what's disabled, and let people override it. Or whatever.

But we need something ...

FediTest slides from #dwebcamp are now on-line:

https://feditest.org/blog/2024-08-08-feditest-at-dwebcamp/

The Fediverse is built on open standard protocols. Testing that implementations conform to those standards is important for interoperability. That's what we are working on with the FediTest project.

Here's a first draft report of Webfinger server conformance for almost 100 Fediverse applications. A total of 1615 tests were run.

Overview: https://feditest.org/contrib/results/2024-06-16/ -- more details by clicking through.

(cont'd...)

ANN: FediTest 0.2 is out. This is our first release, even if early as you can tell from the version number.

feditest is now in the python repos, so you can say "pip install feditest", and the documentation is current on https://feditest.org

You can currently use it to run your own WebFinger server tests similar to https://feditest.org/assets/2024-06-05/webfinger-server-all-wellknown-saas-imp.testmatrix.html Obviously much more to come.

Feedback appreciated!

/cc @activitypubtestsuite #fediverse #activitypub #webfinger #testing #qa

@evan @fedidevs Recording made and will be published shortly.

One of the (many) challenges in #testing the #Fediverse is that some of the standards say one thing, but what is widely implemented is another.

Example: RFC 7033, WebFinger, section 5, requires presence of a CORS header.

So far we've not seen any Fediverse application that actually does this. Arguably it's not needed in the Fediverse, but the standard says MUST.

So we can either have all applications fail that test or not test against the standard. Your thoughts?

https://www.rfc-editor.org/rfc/rfc7033#section-5

FediTest is progressing. Major update this week.

Blog post: https://feditest.org/blog/2024-04-30-update/

Want to try the work-in-progress? https://feditest.org/docs/quickstart/

A FediTest architecture slide made for the IIW session today:

https://feditest.org/blog/2024-04-18-iiw-feditest-architecture/

/cc @activitypubtestsuite

Early(est) access to work in progress on FediTest and the Fediverse test suite now available.

More: https://feditest.org/blog/2024-04-09-earliest-access/

feditest now understands flag --tap to emit test reports in the TAP format, based on request by @evan and others.

Thanks @steve for implementing it!

Feedback from early show-and-tells. Blog post.

https://feditest.org/blog/2024-03-29-early-feedback/

Fediverse testing with FediTest.

Join us: Early show and tell and feedback on March 7, 2024. You are invited!

https://feditest.org/blog/2024-02-28-early-show-and-tell-and-feedback/

Some thoughts on the components needed for the Fediverse Test Suite, and some of the challenges around them.

A new blog post.

https://feditest.org/blog/2024-02-12-components/

What do we mean when we say "Fediverse Test Suite"?

Blogged.

https://feditest.org/blog/2024-02-06-what-is-a-test-suite-for-the-fediverse/

Hi everybody!

Our project to build a Fediverse Test Suite is now officially under way.

And we have a website at https://feditest.org -- it's very rudimentary at this point, and work in progress, but hey, release early and often... :-)

There's no public code yet, but soon there will be, and we'll run some community meetups to show-and-tell and get your feedback. We'll announce when right through this account.