To finish the job, I ordered this little kit which adapts a "coin battery" to fit onto the pads originally occupied by the nasty Varta:
https://www.ebay.com/itm/355497835640
The PCB uses a MOSFET to prevent current from flowing into the non-rechargeable lithium battery when the system is powered.
Older designs used a diode, but the ~0.6V drop across it would bring VCC too close to the lower limit for the Ricoh.