I'm testing Steam Game Recording on Windows now. This is Shadow of the Tomb Raider with the same 1440p/Highest setting as on Linux/Bazzite. Here, you can see the performance hit is only 13% with background game recording ON.
This is the way it *should* be, so I'm convinced Steam on Bazzite is using software encoding and not hardware encoding for this AMD RX 6800M.
I'll definitely be testing this on another more vanilla distro to see how things compare.