Paul Cantrell
The exact meaning of a certain setting in Calendly (app I use w/students for office hours) was really unclear in the UI. I checked the docs, which also were not clear. So I gritted my teeth and clicked the dreaded AI chatbot button. And guess what!
…it was completely unhelpful. (It did find a link to the correct unhelpful documentation, at least.) But lo and behold, it connected me to a human!
…who was also unhelpful for ~5 minutes before they finally understood my question.
1/2
Paul Cantrell
This entire process took ~12 minutes.
Calendly had to pay both some energy-munching LLM and a live human for that time. I find it hard to imagine that this was cost-effective for them vs giving the UX and doc writing, say, 20% more breathing room from the start.
Cost-trimming is so expensive.
2/2
Paul Cantrell
@donaldball
1. You can’t really manage what you can’t measure.
2. You can’t really measure what you want to manage.
People always forget #2.
Donald Ball
@inthehands Like, maybe I could stroke my chin and posit that you'll get a 35% reduction in support case volume and time, but I don't believe it, not really. I don't think those data are clean enough to support that analysis, or could possibly be disentangled from everything else that's changing.
You also aren't gonna see customer happiness in those metrics. You won't see the potential customers that said nah, this isn't good enough, based on the shitty docs.
Donald Ball
@inthehands This is exactly the kind of thing I bemoan about the dAtA-dRiVeN fad that's overtaken tech management over the past decade or so.
I've gone to managers with these concerns and been told, how can you prove e.g. these docs are inadequate, that investing more in tech writers will pay off? Come to me with the metrics and we'll talk.
I can't? I don't think it's possible, not in general? Certainly not a priori.
Paul Cantrell
@aubilenon
I mean, that’s just the thing: vague dissatisfaction’s impact materializes slowly, is hard to measure, and is hard to trace to a root cause. Crappiness always looks cost-effective at first.
Jesse Morris
@inthehands Discount that by the fact that only a tiny percent of their user base is willing to spend even 1 minute figuring this out never mind 12. (Obviously people not understanding how that feature works has its own cost or downside or whatever, but it's pretty hard to account for)
Paul Cantrell
@VisualPlugin
Sure. The setting is a “daily limit,” and it is per event type but not per invitee. I’d wanted a way to prevent one student from (typically in confusion) scheduling multiple appointments in the same day and squeezing others out, but the product can’t do that yet.
VisualPlugin
@inthehands: would you mind specifying what that setting is and what it does?
I'm considering using Calendly in the future.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.