From my perspective, major challenges are:
1. You cannot implement the spec as written and interoperate with mastodon (or any other major fediverse solutions)
2. There isn't an official test suite or a reference implementation. This situation is improving, but it still haunts new implementers
3. Interoperating with anyone is "whackamole driven development" because of the way extensions are handled and what is/is not implemented.
4. There's almost no incremental path.