"A Māori mum misidentified as a trespassed 'thief' at a Rotorua supermarket trialling facial recognition technology says she felt 'racially discriminated' against and embarrassed during the 'horrible' birthday incident."
@strypey > She said she told them: “No way in hell is that’s me, you’ve got my three forms of ID, so you can take your dirty picture and **** off.”
i'm honestly not going to get too mad about them being on alert, but the ID really probably should have cleared it up.
i play with these things they're not 100% accurate. although as Vivek commented during his campaign... when theres a computer in the loop people shut their fucking brains off.
@sun@strypey these things typically use convolution networks to guess at some like 40 facial features, and if you are lucky it goes in to some autoencoder to make a fingerprint of those features, which on a good day will just tell you the face is most similar to something else in the database.
if you ever played with vtube software you can see how they're really jittery too
@icedquinn@strypey you're right and the person couldn't have known if there was AI bias haha. But I'm saying, it should probably be checked so it doesn't keep falsely flagging maori because that sucks
@icedquinn@strypey@sun it's not the issue. The issue is that the software is exactly capable enough to impress executives during a sales meeting and its false-positive rate has been accepted by the company - they don't care
the people making the purchase decision for the software are not, unfortunately, the people designing the system. The system will be designed after the stakeholders have already decided on the technology to use. Engineers only take the output produced by management and assemble it into something as-working-as-possible. Like assembling a lego structure. There is no actual design.
Yes, this is actually how retail development operates.
@icedquinn@strypey@s8n I believe s8n is right that probably as little development goes into it as possible to sell it so it probably works way worse than we think
@s8n@icedquinn@strypey the mistake in thinking that people have is they think that surely, something that affects humans directly was created with much more care. Well, it's not, it's just a "shit it out as fast as possible and make money" "product" like everything else made in America.
@nicholas@icedquinn@strypey yes that is the main reason but a lot of people have dark skin tones, this reality is why accepting these limitations is systemic bias.
I believe the issue with skin tones is just a physical property of the camera needing more light to get the same quality of image from a dark surface compared to a light surface, rather than a systemic bias in the matching software.