Embed Notice

HTML Code

<blockquote style="position: relative; padding-left: 55px;"><section><a href="https://infosec.exchange/users/adulau/statuses/111492488419140109">Alexandre Dulaunoy (adulau@infosec.exchange)'s status on Thursday, 30-Nov-2023 19:18:22 JST</a><a href="https://infosec.exchange/@adulau" title="adulau@infosec.exchange"><img src="https://gnusocial.jp/avatar/204538-48-20231027091710.webp" width="48" height="48" alt="Alexandre Dulaunoy" style="position: absolute; left: 0; top: 0;">Alexandre Dulaunoy</a></section><article><p>Extracting Training Data from ChatGPT</p><p>I’m wondering if OpenAI requested a CVE for the disclosure of this vulnerability.</p><p><a href="https://infosec.exchange/tags/llm" rel="tag">#llm</a> <a href="https://infosec.exchange/tags/llms" rel="tag">#llms</a> <a href="https://infosec.exchange/tags/openai" rel="tag">#openai</a> <a href="https://infosec.exchange/tags/vulnerability" rel="tag">#vulnerability</a> <a href="https://infosec.exchange/tags/chatgpt" rel="tag">#chatgpt</a> </p><p>🔗  <a href="https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html" rel="nofollow noreferrer">https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html</a></p><p>🔗 <a href="https://arxiv.org/abs/2311.17035" rel="nofollow noreferrer">https://arxiv.org/abs/2311.17035</a></p></article><footer><a rel="bookmark" href="https://gnusocial.jp/conversation/2396051#notice-4734608">In conversation</a><time datetime="2023-11-30T19:18:22+09:00" title="Thursday, 30-Nov-2023 19:18:22 JST">Thursday, 30-Nov-2023 19:18:22 JST</time> <span>from <span><a href="https://infosec.exchange/@adulau/111492488419140109" rel="external" title="Sent from infosec.exchange via ActivityPub">infosec.exchange</a></span></span><a href="https://infosec.exchange/@adulau/111492488419140109">permalink</a><h4>Attachments</h4><ol><li><article><header><div>Domain not in remote thumbnail source whitelist: not-just-memorization.github.io</div><h5><a href="https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html">Extracting Training Data from ChatGPT</a></h5><div></div></header><div></div><footer></footer></article></li><li><article><header><div>Domain not in remote thumbnail source whitelist: static.arxiv.org</div><h5><a href="https://arxiv.org/abs/2311.17035">Scalable Extraction of Training Data from (Production) Language Models</a></h5><div></div></header><div>This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset. We show an adversary can extract gigabytes of training data from open-source language models like Pythia or GPT-Neo, semi-open models like LLaMA or Falcon, and closed models like ChatGPT. Existing techniques from the literature suffice to attack unaligned models; in order to attack the aligned ChatGPT, we develop a new divergence attack that causes the model to diverge from its chatbot-style generations and emit training data at a rate 150x higher than when behaving properly. Our methods show practical attacks can recover far more data than previously thought, and reveal that current alignment techniques do not eliminate memorization.</div><footer></footer></article></li></ol></footer></blockquote>

Corresponding Notice

Embed this notice
Alexandre Dulaunoy (adulau@infosec.exchange)'s status on Thursday, 30-Nov-2023 19:18:22 JST Alexandre Dulaunoy
Extracting Training Data from ChatGPT
I’m wondering if OpenAI requested a CVE for the disclosure of this vulnerability.
#llm #llms #openai #vulnerability #chatgpt
🔗 https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html
🔗 https://arxiv.org/abs/2311.17035
In conversationThursday, 30-Nov-2023 19:18:22 JST from infosec.exchangepermalink
Attachments
1. Domain not in remote thumbnail source whitelist: not-just-memorization.github.io
  Extracting Training Data from ChatGPT
2. Domain not in remote thumbnail source whitelist: static.arxiv.org
  Scalable Extraction of Training Data from (Production) Language Models
  This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset. We show an adversary can extract gigabytes of training data from open-source language models like Pythia or GPT-Neo, semi-open models like LLaMA or Falcon, and closed models like ChatGPT. Existing techniques from the literature suffice to attack unaligned models; in order to attack the aligned ChatGPT, we develop a new divergence attack that causes the model to diverge from its chatbot-style generations and emit training data at a rate 150x higher than when behaving properly. Our methods show practical attacks can recover far more data than previously thought, and reveal that current alignment techniques do not eliminate memorization.

Public

Embed Notice

HTML Code

Corresponding Notice