Conversation

Notices

1. Embed this notice
   Simon Willison (simon@fedi.simonwillison.net)'s status on Friday, 29-Nov-2024 08:23:56 JST Simon Willison

   Some notes on running the new SmolVM vision model (~4.2GB) on my MacBook Pro M2 at 75 tokens/second to describe a photo, using a shell one-liner powered by uv and mlx-vlm https://simonwillison.net/2024/Nov/28/smolvlm/

   • Blaise Pabón - controlpl4n3 repeated this.
   • Embed this notice
     Simon Willison (simon@fedi.simonwillison.net)'s status on Friday, 29-Nov-2024 08:24:09 JST Simon Willison

     in reply to

     If you have an Apple Silicon Mac with >24GB of RAM and >5GB of available disk space and uv installed, try running this command to see the new model in action (replace IMG_4414.JPG at the end with a path to your own image)

     uv run \
     --with mlx-vlm \
     --with torch \
     python -m mlx_vlm.generate \
     --model mlx-community/SmolVLM-Instruct-bf16 \
     --max-tokens 500 \
     --temp 0.5 \
     --prompt "Describe this image in detail" \
     --image IMG_4414.JPG