Conversation

Notices

1. Embed this notice
   Mario Zechner (badlogic@mastodon.gamedev.place)'s status on Thursday, 06-Apr-2023 17:14:19 JST Mario Zechner

   So, I've been playing with ChatGPT and consorts in anger for the past 2 weeks.

   Let me show you today's experience with ChatGPT (3.5 turbo).

   I started out by creating a simple, self-contained HTML file with some JS, to parse out a subset of data from JSON. That worked fine, after giving ChatGPT a better idea of the data structure.

   https://sharegpt.com/c/wVi0HAu

   • Embed this notice
     Mario Zechner (badlogic@mastodon.gamedev.place)'s status on Thursday, 06-Apr-2023 17:14:02 JST Mario Zechner

     in reply to

     ChatGPT/Copilot finally give programmers a chance to experience how non-technical people feel when trying to coerce programmers to create a thing.

     You too can now throw imprecise natural language prompts at a magic coding box and get disillusioned by the results.

   • Embed this notice
     Mario Zechner (badlogic@mastodon.gamedev.place)'s status on Thursday, 06-Apr-2023 17:14:05 JST Mario Zechner

     in reply to

     Yes, GPT-4 is slightly better, given the bigger token window. But that's more than counter acted by the further increased response times. Also, it still shits the bed with anything that's "do a qsort in Python".

     This is going to be a tool that will be used by many people, only to realize that diminishes their code comprehension and design skills. We'll have to unlearn all this again, once we see the real world negative impact it has on businesses and software quality.

     pettter repeated this.
   • Embed this notice
     Mario Zechner (badlogic@mastodon.gamedev.place)'s status on Thursday, 06-Apr-2023 17:14:07 JST Mario Zechner

     in reply to

     And even for the "single, self-contained function" use case, it usually shits the bed so hard the further you get away from "standard" problems, you'll spend more time massaging it via natural language input than writing things from scratch yourself.

   • Embed this notice
     Mario Zechner (badlogic@mastodon.gamedev.place)'s status on Thursday, 06-Apr-2023 17:14:08 JST Mario Zechner

     in reply to

     If you think this will replace programmers anytime soon, I would like to get the phone number of your dealer.

     If you give this to a junior dev without human supervision, you will forever have a junior dev churning out bad code that ignores any and all error states and corner cases.

     Response times make this an inefficient tool to work on anything that's longer than a simple, self-contained function.

   • Embed this notice
     Mario Zechner (badlogic@mastodon.gamedev.place)'s status on Thursday, 06-Apr-2023 17:14:11 JST Mario Zechner

     in reply to

     It took me about 25 minutes to get to that stage, mostly because ChatGPT response times are very bad, and because ChatGPT would just keep doing stupid things.

     In that time, I could have written this myself, trice.

     So, my take away is this:

   • Embed this notice
     Mario Zechner (badlogic@mastodon.gamedev.place)'s status on Thursday, 06-Apr-2023 17:14:17 JST Mario Zechner

     in reply to

     My goal was to get to a stage, where I can load 3 files and put the results into a JS map filename -> base64 encoded file content.

     We'll here's the result.
     https://sharegpt.com/c/gmye2Qc

   • Embed this notice
     Mario Zechner (badlogic@mastodon.gamedev.place)'s status on Thursday, 06-Apr-2023 17:14:18 JST Mario Zechner

     in reply to

     Next, I got a little bolder, trying to make ChatGPT load the file(s) and pass it to an existing API, which i tried to describe to it, which coiuld display the file content in a graphical way.

     It was a constant back and forth, without me being able to find a prompt that would make it do what it was asked. It's not a terribly hard use case either. But I figured it's bad at compositing simple tasks like that. So I started from scratch.