Mario Zechner
Today was ... interesting. If you followed me for the past months over on the shitbird site, you might have seen a bunch of angry German words, lots of graphs, and the occassional news paper, radio, or TV snippet with yours truely. Let me explain.
In Austria, inflation is way above the EU average. There's no end in sight. This is especially true for basic needs like energy and food.
Our government stated in May that they'd build a food price database together with the big grocery chains. But..
Mario Zechner
the responsible minister claimed it's an immense task and will take til autumn. It will only include 16 product categories (think flour, milk,etc.). And it will only be updated once a week.
Given how Austria works, some corp close to the minister would have gotten the contract for a million on two to create a POS just enough so the minister can say "look, I did something!"
Well. I heard that and build a prototype for all products of the two biggest chains in 2 hours. The media picked it up...
Mario Zechner
Here's a selection of media coverage of the entire thing.
https://heisse-preise.io/media.html
It spread like wild fire and made the minister look like an idiot.
I took the thing down in fear of retaliation by the grocery chains. My plan: get a big NGO, news outlet or political party to host the thing and be a legal shield for the endevour.
Almost every NGO, media outlet and political party got in contzct with me (not the other way around). There were lots of promises and big words but zero action.
Mario Zechner
The whole thing runs client-site. The server fetches the latest data from the stores once a day. All data fits into 5mb of gzipped JSON. Small enough for the client to do anything. The server just serves 8 static files. It can handle serve all of Austria easily and could be scaled trivially. It's just static files.
Being the idiot I am, I also made it open-source:
https://github.com/badlogic/heissepreise
And as usual, people flocked to it and contributed. In no time we had all stores in Austria in there.
Mario Zechner
All these orgs only had their self-interest in mind. After two weeks of this bullshit, I figured I might as well gamble and put this thing up in my own name.
Surely the grocery chains won't sue me. The bad PR would easily outweigh whatever little inckme loss they'd suffer from a few hundred people using the site to find the cheapest product.
You see, I'm basically just crawling the stores online stores. Most of them have an API. I then normalize the data across the stores, and expose it.
Mario Zechner
I scrambled to integrate his data into my platform. I added analytics tools. And then I ran my first few analyses. And my jaw dropped.
"Well, that's a bit to much of a price increase even given higher energy prices."
So I started to dig. And boy did I find a lot of things...
Mario Zechner
Then we also got German and Slovenian stores. Then we normalized product categories across stores and added some light data science techniques to match the same or similar products across stores to make prices more easily comparable. You know, iterative improvements.
And then some anomymous guy in Twitter send me the data he crawled for the two biggest chains. Starting in 2017. And that's when thinga really got interesting...
Mario Zechner
E.g. given the historical data, I could see price movements for a product across the two chains. And you won't believe what I found (well, you know what's coming...)
Them fine grocery chains changed the prices of the self-branded low cost products with one to two days, or even on the same day. And they both came up with the exact same price.
Mario Zechner
My first analysis actually happened before I build the platform. I was manually comparing prices of products the stores themselves offer in the lowest price segment. Things like grocer store brand milk or flour.
I compared 40 product pairs across the two biggest chains. And lo and behold: their prices matched exactly to the cent!
An NGO picked this up on Twitter and did the analysis for 600 product pairs. Same picture.
With my platform in place, I could do more advanced stuff.
Mario Zechner
We could show shrinkflation, meaning products with less content are sold for the same or even higher price.
Examplified by e.g. laundry detergent.
Mario Zechner
This wasn't only happening in the low-price chain-brand segment. It also happened in the mid-range segment of self-branded goods.
And it all started happening when inflation went through the roof.
Clearly, something was up. My guess was: tacit collusion, meaning, oligopolic price coordination without explicit coordination.
Meanwhile, others have build platforms like I did as well. And they too saw these patterns.
There were more.
Mario Zechner
Even more interestingly, products exclusively produced in Austria cost less outside of Austria.
Billa is the Austrian version of REWEDE.
Even fucking Red Bull, an Austrian brand, costs more in Austria when it is discounted here, than it costs normally without discount in Germany.
WTF.
Mario Zechner
We could also show that the exact same product cost up to 40% less in Germany, a country with higher mean income and higher cost of living.
Mario Zechner
If I were trying to describe it in more flowerly terms: It's asymmetric information war fare.
The stores tell you they are good and benevolent and only have your interest at heart, so here are discounts. Discounts for everyone. They even gamified the whole thing with stickers. I shit you not. People collect stickers they put on the products in the convery belt at the register. There's also apps, which will give them all info on you
In reality it makes it impossible to know how much things cost
Mario Zechner
Then I looked at an aspect pretty unique to Austria: discounts.
You see, in a normal country, with a competitive grocery market, you usually have about 10%-20% of products that get discounted on average.
In Austria, that rate is 40%. It's a fantastic way to obfuscate the actual price of a product. As a customer, you'll never know what you'll pay on that day until you see the current discounts directly in the store.
The chains are very generous and will send you discount leaflets via mail.
Mario Zechner
The grocery chains got a little iffy about all that somewhat negative media coverage, some of which was spurred by my continued analyses.
They started to put out these things in the store. It basically says "We've already lowered the prices of 450 products for you this year". With a sortiment of 22000.
They were also dumb enough to put out a machine readable PDF with all the products they lowered the price for.
With a little data science magic, I was able to match those with my database...
Mario Zechner
Given the historical data I had, I was able to also check for patterns in the discounts they give. How often, how high.
Mario Zechner
All that media coverage of my platform and the platforms of other people, with whom I've started to converse and who've became friends of sorts, triggered the competition authority of Austria.
You know, the guys and gals who's job it is to sniff out anti-competitive behaviour, cartels, price gauging and coordination and so on.
They contacted all of us to ask what we'd need to continue doing our work. They actually saw value in that.
We provided them with a shit ton of feedback.
Mario Zechner
Of course you do, cause you are a smart cookie.
While their claim that they decreased the regular price is correct, they also increased the discounted price that comes into play every other X weeks/days.
So they are again technically correct: the regular price was decreased.
But on average, a consumer pays more if they buy the product every week, as the discounted price has been increased. The average is higher than before.
Sneaky.
Mario Zechner
But there's a more "nefarious" kind of price decrease.
As I said, Austria is a country of insane amounts of cyclic discounts. Many products will be sold for their "regular" price for one week and a discount price the other.
The real price for the consumer is the average of the regular and discounted price.
Given this knowledge, do you notice something with the prices for this product the grocery chain claims to have decreased the regular price on?
Mario Zechner
And lo and behold. There was fun to be had.
There are products that are cyclic in their price changes. E.g. this axe shower gel, which they listed as having a lower price now.
Yeah, you lowered the price from 3.99 to 2.99. But that follows the exact pattern this product's price had over the last couple of years.
Technically correct. But not a permanent price decrease.
Second picture is another example of that.
Mario Zechner
The spot check showed that their claims were true on the surface.
But I'm a stickler for data, so I looked a bit closer.
Mario Zechner
I'm but a lowly computer nerd and lay person, and not someone with an economics degree. I simply handed the data over in the hopes their experts would figure this shit out.
Well. Today they presented their first preliminary report.
In it, they basically copied my long ass email with answers to their questions from earlier more or less verbatim. They agreed with my conclusions regarding what needs to be done on the legal and technical site.
Mario Zechner
The basic gist of that feedback:
- Legal: it must be legal for us to crawl and publish the price data the stores put out on the web in their online stores
- Technical: ideally, stores would be forced to put that data out in a normalized form, so matching and comparisons become easier. We already did that ourselves though, with some data science and heuristics, so no biggie if that doesn't happen.
Besides that feedback, I also send them a shitton of data and patterns I found.
Mario Zechner
The suggestion by the competition authority to the minister was great:
1. Using the data should be made legal by the legislature for certain parties, including price comparison platforms and academic institutions.
2. Grocery chains of a certain size must publish all their data in real-time according to a predefined scheme with all necessary meta data to make things comparable and allow matching of products across stores.
Fantastic! Or so I thought.
Mario Zechner
And they also officially said it's very likely the grocery chains use automated systems to follow each other in prices.
No word on the other data. We'll find out what they think end of October when the full report is scheduled to be released.
Now, here's how the chain of command works in this sector.
The competition authority is apolitical but under the reign of the politically appointed minister of economics. They can only report and suggest to him.
He then decides what gets done.
Mario Zechner
Remember the chain of command. The minister decides what actually gets done.
And that minister is a member of the conservative party. You can already guess what gets done, right?
His plan:
1. The grocery chains must publish data. But only for a hand-picked list of basic products. Not the entire sortiment, like we do now.
2. Platform owners can be sanctioned/sued if they display the data the wrong way.
There's are only two up-sides in all of this.
Mario Zechner
And that was my story. Thank you for coming to my TED talk. And don't spend your holiday money in Austria, we suck.
Mario Zechner
First of all, the minister initially planned to create a price comparison platform "himself". This would have meant that some company he's buddy buddy with would have gotten a million Euro contract and delivered an abmysal failure of a system.
He's now given up on that.
The second upside: as soon as media coverage of our efforts picked up, the price hikes stopped for the most part. I'm obviously not entirely attributing this to our work. But I like to think we played a part in it.
Mario Zechner
Oh, and if you want to do this for your own country, you can re-use what we build so far!
https://github.com/badlogic/heissepreise
Happy to help if you need guidance! Adding a store is usually less than 200 LOC if they have a search API in their web store.
https://github.com/badlogic/heissepreise/blob/main/stores/billa.js
Mario Zechner
Yeah, the irony of sending grocery vouchers for the same grocery stores that I go up against with my platform is not lost on me.
Anyways, we've been able to send out ~4500 vouchers in a bit over a year to as many families. That's about €220,000 worth of donations.
~6000 families have signed up with us, about 1500 are still waiting for a voucher.
If you can spare some money, here you go:
https://cards-for-ukraine.at/donate
The latest batch went out today. CW link to shitbird site
https://twitter.com/badlogicgames/status/1702670312981049561
Mario Zechner
I don't have a sound cloud, but I have another little project.
We have a charity where we ask for donations which we convert into €50 grocery vouchers for Ukrainian families that fled to Austria. Our state fails them as well.
We are zero overhead, every cent goes towards the vouchers. We pay the rest (envelops, stamps, printer cartridges, etc.)
We are 100% transparent, all contracts/orders/bills/payments here:
https://drive.google.com/drive/folders/1PxOL8A44bIRU1Hdoq87_2iXSLNmnMXQr?usp=drive_link
Bunch of friends doing stuff.
Wolf480pl
@badlogic
In Poland we recently got a regulation saying that when advertising a discount, the shop has to provide the lowest price from last 30 days as comparison.
So if 2 weeks ago it was 2.99, yesterday it was 3.99, and today they're lowering the price again to 2.99, they d have to say they lowered from 2.99 to 2.99
Cat poni, lgbt feminist drag nato demon
Mario Zechner
@wtfrank that's why we don't use images.
Francis ☑️
@badlogic good job, well done. I recall that there was a website in the UK that did supermarket price comparisons, but it was shut down via copyright law: the supermarkets licence product images from a third party so when the guy used the same images as the supermarkets to.illustrate the price comparisons, he was threatened by the product image company and forced to shut down.
Mario Zechner
(No the code isn't great, and I should have used TypeScript. I didn't plan for this to become a thing. I'm sorry)
Diabetic Heihachi
@badlogic
Wow, amazing work! This needs to propagate globally. Imagine seeing what patterns of greed and engineered poverty occur on global analytics...
Diabetic Heihachi
@badlogic
I so need to learn more node, what would probably help spread this would be a cleaned up version, no stores setup on install, then a repo the app can access for each store that gets incorporated, folks checkbox and save the stores from their region available, plugin style.
Mario Zechner
@DavBot I'm all for that. That's why this is OSS and easy to self-host. Adding a new store is usually less than 200 LOC (code quality is terrible tho :D)
Diabetic Heihachi
@badlogic
Might help piss enough people off to actually make a difference.
Kristof :nvadmin:
@badlogic Wow .. what a read. Do you know about Project Galileo to help you out protecting the site/servers/... ?
Mario Zechner
@iworx no need for protection. The thing is easily self-hosted. Can even deploy it as a GitHub site, with a daily workflow run scraping the data.
Toni Scott
@badlogic
You have exposed corruption and price fixing in Austria on a massive scale! Well done. Let's hope this, and the indifference of your government has a big influence in your next elections.
Tim
@GhostOnTheHalfShell @2du @badlogic there’s a good Amazon price history / alert site called camelcamelcamel if you haven’t come across it…
GhostOnTheHalfShell
Yeh, cracking Amazon or any of the other platform sites. Air travel would be a doozy.
PassTheSalt
@badlogic I hope that this is possible for the US and that someone picks up on it!!
manu
@badlogic did someone pick this up for Germany yet? Is there a list of platforms working on that for other countries?
Solarpunk Davy
@Datagraver @badlogic I did find a dataset with over a year of data: https://tweakers.net/geek/204304/tweaker-ontwikkelt-website-om-productprijzen-van-supermarkten-te-vergelijken.html
Datagraver.com
@SolarDavy @badlogic Nope.
Great initiative. Great thread to read though!
Solarpunk Davy
@badlogic hey @Datagraver you didn't set this up 3 years ago by accident?
mq86mq
@c3manu @badlogic Beim toten Vogel gibts den #rewebot. Früher hat der kommerzielle https://www.preiszeiger.eu/ die Änderungen in Promillpunkten öffentlich gehabt, woraus man halbautomatisch meistens die absoluten Preise rekonstruieren hat können (ist aber im Gegensatz zu den Händlern eine urheberrechtlich geschützte Datenbank). https://www.discounter-preisvergleich.de/ ist auch fast nur noch historisch und nach offenen Anfängen teils kommerzialisiert worden.
Mario Zechner
@kentindell my data can not on its own. However, for basic groceries, i.e. milk, specific meats, some fruits, another governmental org recently released the prices the grocery chains pay the producers. In that case, it's the stores that did the gouging.
We do observe stagnant/decreased year-over-year profits for grocery chains in Austria as well. But there's a big but :)
You see, their operating expanses have not increased dramatically. What has increased are their "investments".
Ken Tindell
@badlogic That was an absolutely fascinating thread. In the UK there are complaints about prices, but a quick look at the profitability of supermarkets shows they’re not benefiting from this overall - their profits have gone down. But the suppliers of branded products to supermarkets have been taking the opportunity to increase their margins. Can your data processing observe this retailer/supplier pricing effect?
Mario Zechner
@kentindell where "investments" are things like building new stores or parking the money in financial assets. Austria has the highest density of grocery stores in all of Europe. There's absolutely no need for this density.
This little trick allows them to lower their official "profit" numbers in their financial reports.
The UK fucked itself with Brexit when it comes to food prices, so your situation may actually be caused by the price increases of out of country producers.
SamuelJohnson
@badlogic @kentindell UK has waived import controls for the 5th time on food coming from the EU and I predict controls will not happen before the next election--when the current govt will be soundly defeated. Why: prices would go up and many EU exporters would just stop bothering w GB (no problem for NI). Nor will it happen afterwards bc next govt will either rejoin single market / customs union or agree on standards. Turns out you can't eat a 🇬🇧 flag. Who knew?
Brilliant work btw 👏👌💪
Mario Zechner
@kentindell iirc, the UK sources a lot of food from countries like Spain. With Brexit, that became significantly more expensive. Not an expert on that tho, take this with a table spoon of salt.
Veza85UE
@badlogic
I'll do my best to somehow not spend any money on groceries when I change trains in Salzburg next month, will stock up in (politically-impeccable) Italy and just have breakfast. 😄 Good to see Austrians joining Romanians, Bulgarians, Salvini, Selmayr and half of Europe basically in being mad at Austrian politicians treating their voters like dummies.
(Best thread I've read on Mastodon so far, btw. 👏)
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.