Conversation

Notices

1. Embed this notice
   Isaac Lyman (isaaclyman@toot.cafe)'s status on Monday, 04-Nov-2024 06:28:32 JST Isaac Lyman

   I am once again calling for a public university to develop an open-source, publicly-accessible internet search index.

   Search is a public good.

   #opensource

   • Embed this notice
     Isaac Lyman (isaaclyman@toot.cafe)'s status on Monday, 04-Nov-2024 06:28:30 JST Isaac Lyman

     in reply to

     The university that does this will have an unending treasure trove of capstone and thesis projects for students in technical majors. Search is a thousand hard CS/DS/IT problems and cat-and-mouse games combined into one. Is it enough to justify the cost? No. But with a subsidy, it certainly wouldn’t be a chore.

     #opensource

     Jorge Stolfi and Xenotar repeated this.
   • Embed this notice
     Isaac Lyman (isaaclyman@toot.cafe)'s status on Monday, 04-Nov-2024 06:28:30 JST Isaac Lyman

     in reply to

     I’m excited to hear that opensearchfoundation.org is working on this exact problem, though they’re focusing their efforts on Europe exclusively. Worth a read of their website; they’re funded by a grant from the EU and are technologically competent. They’ve crawled 2 billion URLs so far and are well on their way to a 2025 delivery date.

     My guess is the index files will go somewhat viral and OSF-based search will be available in North America as well.

     https://openwebsearch.eu/the-project/

     #opensource

   • Embed this notice
     Isaac Lyman (isaaclyman@toot.cafe)'s status on Monday, 04-Nov-2024 06:28:31 JST Isaac Lyman

     in reply to

     To be clear, I’m calling for an open source *index*, not another search engine.

     Practically all search engines (whether for-profit or nonprofit) are renting results from Google and/or Microsoft. The indexes maintained by both companies are driven by anti-consumer motives.

     If there were a stable, financially independent index available to the public, we’d have our choice of for-profit, nonprofit, and FOSS engines built on top of it.

     #opensource

   • Embed this notice
     Isaac Lyman (isaaclyman@toot.cafe)'s status on Monday, 04-Nov-2024 06:28:31 JST Isaac Lyman

     in reply to

     Building, hosting, and maintaining a performant search index is expensive. We likely will not ever see a nonprofit come up with the resources to do this. But the internet is an indispensable pillar of infrastructure, and is extraordinarily difficult to use without search.

     Economically speaking, the right solution is to treat search as a public good and pay for it with our taxes.

     #opensource