John Mueller
I'd like to better understand the reluctance about full #search on #Mastodon. Is there an overview somewhere?
Working at Google on search, and with SEOs, I'm not used to folks publishing things publicly, and then not wanting them to be easily found. Is there something Google could be doing - in search or in the form of guidance - that would help keep folks comfortable, and still be useful?
Tokyo Outsider (337ppm)
@kiki @katzentratschen @johnmu Best explanation I've seen. ☝️ There's a reason the regulars all stop talking when strangers walk into a small pub.... even though that pub is a "public" space.
Kiki
@katzentratschen @johnmu It‘s like talking with a friend on a park bench or on the bus. It‘s a conversation that takes place in the public space, but it‘s not a public conversation. You don‘t really mind strangers overhearing parts of it while walking by, but if you catch them stopping next to you only to listen in, you‘ll be somewhat irritated.
John Mueller
@katzentratschen Thanks, very helpful!
One of the things I'm trying to understand better is why the general feeling is the content shouldn't be public to the world, and yet it is published as such.
To me, making that explicit (eg, a login-wall) would set clearer boundaries, but I realize I definitely don't know the whole history.
Katzentratschen
@johnmu You've already read @Gargron's post?
https://blog.joinmastodon.org/2018/07/cage-the-mastodon/
Another post from my bookmarks:
https://www.hughrundle.net/home-invasion/
Moreover you could follow Hashtags like #FediSearch and #FediCulture and take a look at GitHub.
Luca
@johnmu You might want to have a look at the discussions around https://fedsearch.io/ which offered results from a global search across instances and was then stopped due to backlash from the community. https://infosec.exchange/@jerry/109311187050279476
Daniel Schildt
@johnmu Most people don't understand how to limit search engine indexing. But it's already possible to limit from the Mastodon settings (Preferences, Other, Opt-out of search engine indexing).
"Affects your public profile and post pages"
Sadly people don't adjust the default settings and blame search engines (instead of the default settings of Mastodon software). By default every post on Mastodon is publicly indexable.
Tokyo Outsider (337ppm)
@Luca @johnmu This is exactly what I mentioned just now in another offshoot of this thread. I don't understand why John is saying no protocol exists for this and it's unclear how Google should handle this content when it's actually very clear how Google should handle this content and a protocol has existed for almost as long as the internet itself — certainly for the last 25 years. How could someone "work on search at Google" and not know this?
Luca Hammer
@johnmu Mastodon has a search engine opt out setting for each profile which sets a noindex meta tag (https://vis.social/@Luca/109620741021800649). Honoring that and the robots.txt should be enough in my opinion. But as you have seen, many people see that differently.
The majority of people I reach are fine with hashtags in public posts being indexed: https://vis.social/@Luca/109619582185050938
Tokyo Outsider (337ppm)
@katzentratschen @kiki These could be great addition. The issue with replies by people who opted out being indexed because they're replying to someone who hasn't could also use some work to make sure people know what is and is not being indexed, too — maybe there could be a notice when replying to a post that will be indexed. There are also some issues with deletion that I've flagged on Github. cc: @Gargron
Katzentratschen
@tokyo_0 @kiki @johnmu Definitely. Therefore I'd prefer additional settings: "mentioned only", "followers only", NEW "local only", NEW "local and federated", "local and federated but unlisted", and finally "public" in a globally public sense. The old "public" should translate to "local and federated" to prevent old posts getting more visibility than intended.
Tokyo Outsider (337ppm)
@johnmu To avoid duplication for all concerned, here's the thread https://mstdn.social/@tokyo_0/109624217124137223
John Mueller
@tokyo_0 This setting only applies to new top level posts that a person does -- all replies may be indexable regardless, depending on who you're replying too (and the UI does not show you where this is the case).
Also, the way robots.txt & robots meta tags are implemented here explicitly allows public, full-text search for all public content. Search engines follow those rules, and if there's something deemed useful, they will try to index it if it's annotated as being public.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.