@corbet hey, I wrote you a short thing: https://git.sr.ht/~gnomon/fetch-goo.gl-shortlink-dereferences
It turns out that the ~19,000 goo.gl shortlinks in that lore search you posted deduplicate down to about 360 unique shortlinks. The script in that repo can pull down about 285 of them. Stuffs 'em in a smol sqlite3 database with a simple index that makes lookups, even in a tight loop, close to instant.
It's only the very simplest proof of concept, but it _does_ work. The Lore picture is not as bad as I expected.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.