Strypey
@douginamug
> cool to read your wiki bio, thanks for your struggle!
Thanks a lot for making the effort to say so. I get a lot of job satisfaction, but I only get paid for a tiny slice of my work (I mostly survive on welfare benefits). So hearing from people who value what I do means a lot to me.
Doug Webb
@strypey thanks! And cool to read your wiki bio, thanks for your struggle!
Strypey
@douginamug
> what is the Kalashnikov principle?
See;
Doug Webb
@strypey what is the Kalashnikov principle?
Strypey
@douginamug
> Just worth knowing they exist
For sure. For Kalashnikov Principle reasons if nothing else.
Doug Webb
@strypey I hope so too. Just worth knowing they exist, and seems you do.
Strypey
@douginamug
> DYK Kagi?
That's the search engine that charges for access? Good luck to them, but I can't see this ever being more than a niche service, for people with money to spare. I think we can come up with models that solve the web search problem in new ways, and can be financially sustainable without paywalls.
Doug Webb
@strypey DYK Kagi?
Strypey
Having such data available might have unintended consequences. Some of them related to the other reasons people scrape the web;
https://pluralistic.net/2023/09/17/how-to-think-about-scraping/
If anyone can think of any, I'm curious about what they are.
(9/9)
Strypey
Perhaps sites could create their own index, and update it every time the site contents change? Or maybe for busy sites, in daily batches. Then offer copies of that index using a standard format and address (eg domain.url/search).
Search portals could download index files at an agreed rate. Eg once a day for busy sites, once a week for less busy ones, and 3 monthly for static sites that only change occasionally.
I'm sure I've ranted about this before. How could this approach be tested?
(7/?)
Strypey
There are search portals like searchmysite.net, that like Wikia Search use, a human-curated search index;
https://the.socialmusic.network/t/marginalia-search-engine/167/2
But presumably what the humans are curating is which sites are included in the index, while automated scrapers do the actual indexing of site contents.
Couldn't they reduce the compute used on their servers, and the scraped site's webserver, by having the site index itself? So the search portal only downloads 1 data file, instead if every page on the site?
(8/?)
Strypey
Even the best human curation has limits though, and we know web indexing can be automated. But we'd want search portals doing it in a repeatable and verifiable way, with the cooperation of web maintainers, at the lowest possible resource cost to both.
(6/?)
Strypey
People have been bouncing around ideas for replacing Goggle Search since the launch of G+, when the enshittification of everything Goggle began in earnest. The most obvious one is the Wikipedia model, the same, centralised search portal, but run by a not-for-profit, and funded by donations and grants rather than advertising.
People sometimes say it's been tried, by Jimmy Wales, with Wikia Search. But Wikia was a for-profit startup, not a not-for-profit.
(4/?)
Strypey
Plus, I don't think 1 failed prototype proves a model isn't viable. From memory, Wikia Search depended entirely on building a human-curated index, but launched to the public very early in that process. Probably not surprising then that the search experience was less than impressive, compared to the Goggle Search of 2008.
The other limitation of Wikia's search model was that depending on 1 group of link reviewers. How about social portals sharing their search index updates over a network?
(5/?)
Strypey
I come back to the future of web search often, because it fascinates me, and because it's important. Here's why.
The Open Web doesn't work as a mass medium without effective discovery tools. The existing ones are rapidly enshittifying, both search and social platforms. Worse, almost everyone - including TBL - is hypnotized by a future of Trained #MOLEs doing our information processing for us.
So those who are not, and still think the Open Web matters, need to get our thinking caps on.
(3/?)
Strypey
So if we *have to* replace Goggle.com for web search, because that's no longer what it does, what do we want to replace it with? How could it be structured? And the million dollar question - because the answer to this inevitably shapes the structure - how could it be funded?
(2/?)
Strypey
When Goggle.com started replaced its search engine with a Trained #MOLE, it started a transition from being a web search engine, to becoming a *guessing* engine. By replacing the list of webpage links - the output of a web search engine - with a chatbot interface that outputs word salad, Goggle are just completing a process they started years ago.
(1/?)
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.