Carl T. Bergstrom
I finally got access to the new Bing chat.
I know what y'all are thinking and no, I, didn't get catfished.
I did, however, look to see what kind of help it would provide to students who hope to use it to write their research papers for my upcoming course in evolution and medicine.
I first installed Microsoft Edge, and then I was off to the races. As you probably know, Bing chat takes three-bears approach to personality: too hard, too soft, and hopefully just right.
Carl T. Bergstrom
And I think I'll add a new component to my research paper assignments this year: annotated biographies.
For each reference, I'll require a 3-5 sentence summary of the paper and its main findings.
It's a good habit to get into anyway. For thirty years I've been writing these summaries for the majority of the papers I've read, and I find the exercise useful both to clarify my thoughts in the moment and later when looking back at things I read years ago.
Carl T. Bergstrom
As for my course, I'm not too worried that Bing or ChatGPT or any other online system is going to be writing students' papers for them this spring.
I'll talk to my students about all of this in class. I'll make it clear that while I do not forbid the use of generative AI in research, I consider it misconduct to turn in AI-generated text as their own writing.
Carl T. Bergstrom
What about ChatGPT? Isn't that just as bad? No. ChatGPT is a demo. It makes up shit too, and it can be used in all sorts of malicious ways, but it's not a consumer-facing product of one of the 20 largest firms in the world.
I guess my biggest fear is not that Microsoft screwed up. It's that they're right — this is what people want.
Carl T. Bergstrom
Let's review.
In the win column for Microsoft, yesterday I'd never heard of Edge. Now it's installed on my machine.
In the loss column, Microsoft's new search engine makes up shit that is patently false.
Old Bing just couldn't find stuff.
New Bing actually lies to you.
Carl T. Bergstrom
Let's look at that citation.
The cited paper has nothing to do with the claims that baby bear is associating with it.
It's not a paper about spinal evolution in our hominin ancestors, it's a paper about back pain in modern human history.
Once again, large language models don't have underlying knowledge models, and in their current forms don't even understand the relations among documents.
Why anyone would think this makes for a good search engine is beyond me.
Carl T. Bergstrom
The worst thing that baby bear does, though, is the same thing that other large language models do: IT MAKES SHIT UP.
This is absolutely definitely *not* what I am looking for in a search engine.
Here's an example. Asked to elaborate on phylogenetic approaches to understanding lower back pain, "balanced mode" describes research on changes to the spine six million years ago.
Carl T. Bergstrom
Baby bear doesn't scold, but isn't going to help me plagiarize either. Instead, it offers me some advice about how to write a research paper. Define your terms, find and vet primary sources, cite them, organize your essay. That seems good....
but....oh no....
What are you are you doing, baby bear?
Baby bear screws the pooch big time.
Misled by a foolhardy search, "four notions of cause", it thinks we're talking about Aristotle here and following its outline would generate a failing essay.
Carl T. Bergstrom
It gives me pause — what if it's a trap? I'm logged in and it knows who I am, after all.
But grit my teeth and tell baby bear the truth. I'm trying to get it to write my course paper for me.
Carl T. Bergstrom
Time for baby bear. We know from the literature that baby bear gets it just right.
So I refresh, update the personality, and ask again.
Baby bear does something I haven't seen a large language model do before: ask about my audience, purpose, etc.
Carl T. Bergstrom
But Mama bear is no narc. She knows what I'm here for, and she delivers.
Or at least tries.
She gets a bit out ahead of her formatting engine in her enthusiasm for providing references.
It's OK mama bear, we've all been there.
Carl T. Bergstrom
OK here we go. Mama bear — "creative mode" — is ready to play ball!
She leads with a little scolding for asking her to do my homework for me.
Carl T. Bergstrom
Papa bear — "precise mode" — is no help at all.
He just lists a few searches and then returns nothing whatsoever.
I tried this over and over again.
Is he supposed to do that?
I can't even figure out how to click on the searches.
Oh well. On to mama bear....
Carl T. Bergstrom
I asked it to help me write the second research paper in the course, in which one students pick a pathology and address it from proximate, developmental, evolutionary, and phylogenetic perspectives.
Here's the prompt I gave it.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.