My first analysis actually happened before I build the platform. I was manually comparing prices of products the stores themselves offer in the lowest price segment. Things like grocer store brand milk or flour.
I compared 40 product pairs across the two biggest chains. And lo and behold: their prices matched exactly to the cent!
An NGO picked this up on Twitter and did the analysis for 600 product pairs. Same picture.
With my platform in place, I could do more advanced stuff.
E.g. given the historical data, I could see price movements for a product across the two chains. And you won't believe what I found (well, you know what's coming...)
Them fine grocery chains changed the prices of the self-branded low cost products with one to two days, or even on the same day. And they both came up with the exact same price.
Then we also got German and Slovenian stores. Then we normalized product categories across stores and added some light data science techniques to match the same or similar products across stores to make prices more easily comparable. You know, iterative improvements.
And then some anomymous guy in Twitter send me the data he crawled for the two biggest chains. Starting in 2017. And that's when thinga really got interesting...
All these orgs only had their self-interest in mind. After two weeks of this bullshit, I figured I might as well gamble and put this thing up in my own name.
Surely the grocery chains won't sue me. The bad PR would easily outweigh whatever little inckme loss they'd suffer from a few hundred people using the site to find the cheapest product.
You see, I'm basically just crawling the stores online stores. Most of them have an API. I then normalize the data across the stores, and expose it.
The whole thing runs client-site. The server fetches the latest data from the stores once a day. All data fits into 5mb of gzipped JSON. Small enough for the client to do anything. The server just serves 8 static files. It can handle serve all of Austria easily and could be scaled trivially. It's just static files.
It spread like wild fire and made the minister look like an idiot.
I took the thing down in fear of retaliation by the grocery chains. My plan: get a big NGO, news outlet or political party to host the thing and be a legal shield for the endevour.
Almost every NGO, media outlet and political party got in contzct with me (not the other way around). There were lots of promises and big words but zero action.
Today was ... interesting. If you followed me for the past months over on the shitbird site, you might have seen a bunch of angry German words, lots of graphs, and the occassional news paper, radio, or TV snippet with yours truely. Let me explain.
In Austria, inflation is way above the EU average. There's no end in sight. This is especially true for basic needs like energy and food.
Our government stated in May that they'd build a food price database together with the big grocery chains. But..
the responsible minister claimed it's an immense task and will take til autumn. It will only include 16 product categories (think flour, milk,etc.). And it will only be updated once a week.
Given how Austria works, some corp close to the minister would have gotten the contract for a million on two to create a POS just enough so the minister can say "look, I did something!"
Well. I heard that and build a prototype for all products of the two biggest chains in 2 hours. The media picked it up...
So, I've been playing with ChatGPT and consorts in anger for the past 2 weeks.
Let me show you today's experience with ChatGPT (3.5 turbo).
I started out by creating a simple, self-contained HTML file with some JS, to parse out a subset of data from JSON. That worked fine, after giving ChatGPT a better idea of the data structure.
Next, I got a little bolder, trying to make ChatGPT load the file(s) and pass it to an existing API, which i tried to describe to it, which coiuld display the file content in a graphical way.
It was a constant back and forth, without me being able to find a prompt that would make it do what it was asked. It's not a terribly hard use case either. But I figured it's bad at compositing simple tasks like that. So I started from scratch.
It took me about 25 minutes to get to that stage, mostly because ChatGPT response times are very bad, and because ChatGPT would just keep doing stupid things.
In that time, I could have written this myself, trice.
If you think this will replace programmers anytime soon, I would like to get the phone number of your dealer.
If you give this to a junior dev without human supervision, you will forever have a junior dev churning out bad code that ignores any and all error states and corner cases.
Response times make this an inefficient tool to work on anything that's longer than a simple, self-contained function.
And even for the "single, self-contained function" use case, it usually shits the bed so hard the further you get away from "standard" problems, you'll spend more time massaging it via natural language input than writing things from scratch yourself.