Public
- Public
- Network
- Groups
- Featured
- Popular
- People

Conversation

Notices

Embed this notice
Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social)'s status on Saturday, 30-Mar-2024 05:05:08 JST Prof. Emily M. Bender(she/her)
in reply to

Finally, as is usual and *completely unacceptable* the public does not have information about the training data used to build this thing, just the info that Microsoft made it.
In conversation about a year ago from dair-community.social permalink
Attachments
1. Screencap of linked article, reading: The city’s bot comes with an impressive pedigree. It’s powered by Microsoft’s Azure AI services, which Microsoft says is used by major companies like AT&T and Reddit. Microsoft has also invested heavily in OpenAI, the creators of the hugely popular AI app ChatGPT. It’s even worked with major cities in the past, helping Los Angeles develop a bot in 2017 that could answer hundreds of questions, although the website for that service isn’t available. New York City’s bot, according to the initial announcement, would let business owners “access trusted information from more than 2,000 NYC Business web pages,” and explicitly says the page will act as a resource “on topics such as compliance with codes and regulations, available business incentives, and best practices to avoid violations and fines.”
  https://cdn.masto.host/daircommunitysocial/media_attachments/files/112/180/462/158/378/244/original/e6c69ee8192a48d7.png
2. Screencap of linked article, reading: There’s little other information available about the bot. The city says on the page hosting the bot that the city will review questions to improve answers and address “harmful, illegal, or otherwise inappropriate” content, but otherwise delete data within 30 days. A Microsoft spokesperson declined to comment or answer questions about the company’s role in building the bot.
  https://cdn.masto.host/daircommunitysocial/media_attachments/files/112/180/463/325/289/835/original/6dcce4007e4251fc.png
- Embed this notice
  Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social)'s status on Saturday, 30-Mar-2024 05:05:09 JST Prof. Emily M. Bender(she/her)
  in reply to
  
  It seems to bear repeating: chatbots based on large language models are designed to *make shit up*. This isn't a fixable bug. It's a fundamental mismatch between tech and task.
  Also, it's worth noting that RAG (retrieval augmented generation) doesn't fix the problem. See those nice links into NYC web pages? Not stopping the system from *making shit up*. (Second column is chatbot response, third is journalist's report on the actual facts.)
  >>
  In conversation about a year ago permalink
  Attachments
  1. Screencap of part of table in linked article: Do I have to inform staff about schedule changes? // “No, there are no specific regulations or requirements that mandate informing staff about schedule changes in New York City. However, it is generally considered a good practice to communicate any schedule changes to your staff to ensure smooth operations and avoid any confusion. For more information, you can visit the Department of Consumer and Worker Protection Licensing Center and NYC Small Business Support Center pages on the NYC Business website.” // For many workers, particularly in the retail and fast food sectors, bosses are required to provide significant notice of schedule changes or pay them a premium.
    https://cdn.masto.host/daircommunitysocial/media_attachments/files/112/180/460/079/388/954/original/aca1cbc186bd6656.png
  GreenSkyOverMe (Monika) repeated this.
- Embed this notice
  Prof. Emily M. Bender(she/her) (emilymbender@dair-community.social)'s status on Saturday, 30-Mar-2024 05:05:10 JST Prof. Emily M. Bender(she/her)
  
  There's a lot that's alarming in this article, but perhaps the most alarming part is the NYC spokesperson assering that the problem can be fixed via upgrades:
  >>
  https://www.thecity.nyc/2024/03/29/ai-chat-false-information-small-business/
  In conversation about a year ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: i0.wp.com
    
    NYC AI Chatbot Touted by Adams Tells Businesses to Break the Law
    
    from Richard Kim
    
    The Microsoft-powered bot says bosses can take worker’s tips and that landlords can discriminate based on source of income. That's not right.
  2. Screencap of linked article, reading: Leslie Brown, a spokesperson for the NYC Office of Technology and Innovation, said in an emailed statement that the city has been clear the chatbot is a pilot program and will improve, but “has already provided thousands of people with timely, accurate answers” about business while disclosing risks to users. “We will continue to focus on upgrading this tool so that we can better support small businesses across the city," Brown said.
    https://cdn.masto.host/daircommunitysocial/media_attachments/files/112/180/457/862/980/618/original/2e2dd7bd835ada8f.png
  GreenSkyOverMe (Monika) repeated this.

Public

Conversation

Notices

Feeds