Embed Notice

HTML Code

<blockquote style="position: relative; padding-left: 55px;"><section><a href="https://mastodon.social/users/counternotions/statuses/113545380337750228">counternotions (counternotions@mastodon.social)'s status on Tuesday, 26-Nov-2024 05:18:23 JST</a><a href="https://mastodon.social/@counternotions" title="counternotions@mastodon.social"><img src="https://gnusocial.jp/avatar/98428-48-20230214142051.webp" width="48" height="48" alt="counternotions" style="position: absolute; left: 0; top: 0;">counternotions</a></section><article><p>The incredible lightness of ‘weightless bits and bytes’. <br>↓</p></article><footer><a rel="bookmark" href="https://gnusocial.jp/conversation/4068966#notice-7951511">In conversation</a><time datetime="2024-11-26T05:18:23+09:00" title="Tuesday, 26-Nov-2024 05:18:23 JST">about 7 months ago</time> <span>from <span><a href="https://mastodon.social/@counternotions/113545380337750228" rel="external" title="Sent from mastodon.social via ActivityPub">mastodon.social</a></span></span><a href="https://mastodon.social/@counternotions/113545380337750228">permalink</a><h4>Attachments</h4><ol><li><label><a rel="external" href="https://gnusocial.jp/attachment/3566946">“Today’s largest AI clusters use around 100,000 Nvidia H100 chips and consume roughly 100 megawatts of power. Next year's clusters will scale to 300,000 to 500,000 chips requiring nearly a gigawatt of power about the consumption of a small city,” the firm said. That’s imposing real physical constraints, as “the tech giants who built empires on weightless bits and bytes are now grappling with atoms steel, copper, water rights, and critically, natural gas,” according to the firm.
[…]
“Apple has quietly published research showing how to run large language models directly on devices with limited memory. By leveraging novel storage techniques and the inherent sparsity in AI models, the company can reduce memory requirements by 50% while maintaining performance. Advances like these could enable future iPhones and Macs to run sophisticated AI models locally and privately a development that could radically reshape the entire Al infrastructure landscape,” the firm said.</a></label><br><a href="https://files.mastodon.social/media_attachments/files/113/545/379/257/010/733/original/b006945cf610fdaf.jpg" rel="external">https://files.mastodon.social/media_attachments/files/113/545/379/257/010/733/original/b006945cf610fdaf.jpg</a></li></ol></footer></blockquote>

Corresponding Notice

Embed this notice
counternotions (counternotions@mastodon.social)'s status on Tuesday, 26-Nov-2024 05:18:23 JST counternotions
The incredible lightness of ‘weightless bits and bytes’.
↓
In conversation7 months ago from mastodon.socialpermalink
Attachments
1. “Today’s largest AI clusters use around 100,000 Nvidia H100 chips and consume roughly 100 megawatts of power. Next year's clusters will scale to 300,000 to 500,000 chips requiring nearly a gigawatt of power about the consumption of a small city,” the firm said. That’s imposing real physical constraints, as “the tech giants who built empires on weightless bits and bytes are now grappling with atoms steel, copper, water rights, and critically, natural gas,” according to the firm. […] “Apple has quietly published research showing how to run large language models directly on devices with limited memory. By leveraging novel storage techniques and the inherent sparsity in AI models, the company can reduce memory requirements by 50% while maintaining performance. Advances like these could enable future iPhones and Macs to run sophisticated AI models locally and privately a development that could radically reshape the entire Al infrastructure landscape,” the firm said.
  https://files.mastodon.social/media_attachments/files/113/545/379/257/010/733/original/b006945cf610fdaf.jpg

Public

Embed Notice

HTML Code

Corresponding Notice