Conversation

Notices

1. Embed this notice
   Devin Prater :blind: (devinprater@tweesecake.social)'s status on Tuesday, 14-Nov-2023 08:08:02 JST Devin Prater :blind:

   I wonder if Retroarch could be configured to send a screenshot of a video game and then have that converted into a machine-readable image of items around the screenshot, and the player's position, and then allow the player to choose an item to navigate to from there? Just like, a random idea. I don't know how much GPT4 vision API makes the screenshot smaller or downscales it.

   #accessibility #ai #gpt #blind

   • Embed this notice
     gnutelephony (gnutelephony@floss.social)'s status on Tuesday, 14-Nov-2023 08:08:01 JST gnutelephony

     in reply to

     @devinprater This made me think a little outside the box. Could we create a blind asteroid game using 3d false positioning audio so one can "hear" the apparent position, distance, and direction of incoming asteroid rocks as they approach? I actually once had a lab setup to investigate these kinds of questions as a way to improve audio conferencing, too, but nobody cared then either.

   • Embed this notice
     gnutelephony (gnutelephony@floss.social)'s status on Tuesday, 14-Nov-2023 20:52:02 JST gnutelephony

     in reply to

     • wizzwizz4

     @wizzwizz4 @devinprater perhaps now, but this was 30 years back...to make an asteriod audio game work very realistically you need to consider doppler shift too, as well as frequency profiles (fft) biases defined by the shape and relative angle of the ear canal of each ear relative to the sound source, too, for example, as well as separation distance of the ears. There are many aspects to perception of sound audio position and motion.

   • Embed this notice
     wizzwizz4 (wizzwizz4@fosstodon.org)'s status on Tuesday, 14-Nov-2023 20:52:03 JST wizzwizz4

     in reply to

     • gnutelephony

     @gnutelephony @devinprater In theory, most OSs should support it these days. Can't find a good example, unfortunately, but the web does it: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Web_audio_spatialization_basics

   • Embed this notice
     wizzwizz4 (wizzwizz4@fosstodon.org)'s status on Tuesday, 14-Nov-2023 21:48:08 JST wizzwizz4

     in reply to

     • gnutelephony

     @gnutelephony @devinprater Yeah, I'm not sure how the current systems take into account individual ear geometries when they're doing the "fake surround sound using headphones" thing. But the marketing copy says it works, so they must be doing something!

   • Embed this notice
     gnutelephony (gnutelephony@floss.social)'s status on Tuesday, 14-Nov-2023 21:48:08 JST gnutelephony

     in reply to

     • wizzwizz4

     @wizzwizz4 @devinprater at the time we actually made Styrofoam "heads" that we carved ear canals and placed mics into, and a sweep audio source we could move and position around in a sound studio, to get a basic idea. Sometimes the best way to figure something out is a physical model.