So the Palm Zire 71 is my number one recommendation for a low-fi early 2000s digital camera.
It takes SD cards which makes transfer to modern computer super easy, it is very compact and often sellers don’t know it has a camera (it’s sort of a hidden feature!) so it’s cheap. You can find replacement batteries for it too for cheap.
The photos have an amazing aesthetic.