Blog post: An illustration of how much X cares about memory usage https://utcc.utoronto.ca/~cks/space/blog/unix/XServerBackingStoreOptional tl;dr: if another window covers up part or all of your window, the X protocol allows the server to throw away those parts of your window, stop you drawing new things to them, and then require you to repaint them later, all so that the server doesn't have to allocate an off-screen buffer for your pixels.
(X clients used to render 'directly' to the screen, not to an intermediate buffer.)
The next bit of the cursed X Window System knowledge show will be about how it's all windows, all the way down. So many windows, at least if you write X clients the way that the designers of X expected you to, instead of how everyone does it today.
(That it's windows all the way down intersects with how X supports and sort of assumes server side graphics rendering.)
@quodvideo Hardware acceleration of drawing primitives (2D and 3D) was another area where the X server lagged (and was slow). Everyone wanted direct to hardware OpenGL if they were doing OpenGL at all.
Another driver was fonts and proper text rendering, which basically has to be client side and shipped as bitmaps.
Once you're doing client side rendering, the tree of subwindows approach is relatively terrible (or basically off the table).
@quodvideo I suspect that one part of client side rendering is that general rendering iterated faster than X protocol extensions were able to as CPU and memory improved. If you render client side and ship bitmaps, you can do whatever alpha blending and complex shapes and so on you want, without waiting for an X protocol extension. (And you can render with uniform code for things that partially involve server-unsupported bits, like SVGs.)
Blog post: A peculiarity of the X Window System: Windows all the way down https://utcc.utoronto.ca/~cks/space/blog/unix/XWindowsAllTheWayDown tl;dr: the original idea of how you would program X apps is that most UI widgets and elements would be protocol-level '(X) Window' objects, in a whole tree of them. This was likely designed to allow various clever tricks to reduce client/server traffic, but it pretty much required server side rendering (of basic X graphic elements). It's now fallen by the wayside.
@cks It's never been clear to me why the tree of windows method went away. It always felt like the toolkit developers just decided they didn't like X and weren't going to go along with the design. It's weird to me because boring old widgets work fine the X way and client side rendering could be saved for things needing more detail, like web pages, PDFs, CAD, etc.
Blog post: The X Window System and the curse of NumLock https://utcc.utoronto.ca/~cks/space/blog/unix/XNumlockCurse tl;dr: having NumLock on can more or less invisibly break key bindings because 'NumLock' is treated more or less like a modifier such as 'Shift'. And things can turn NumLock on for you, because they feel helpful. All of this is tangled up in how X turns keycodes into 'what they mean', or doesn't.
(Maybe X could do with another layer of translation; keycode → keycap label → actual meaning considering modifiers.)
Today's cursed X Windows System knowledge is about how X has two cut and paste systems and they don't interoperate, because after people used the first system for a while they realized it had limitations and was kind of a bad idea but of course it couldn't be replaced. Do all programs support both? No, of course not.