Erlend Sogge Heggen> (…) split the index, which is to say the part of Google Search that scrapes the web and makes that content searchable, from the search user interface, and manage that index as a public utility that different search services could rely on and pay for, an idea that was suggested in a recent paper.
That would be incredible. There’s also an efficient-resourcing argument to be made here, because many sites are hit hard by swarms of crawlers, all collecting the same data.
https://mastodon.social/@robin/112933151964297678
All GNU social JP content and data are available under the