It listens, it looks, it answers

An assistant that takes a spoken question and an image in the same turn and answers both at once. Two minutes of it running, recorded March 17, 2024.

The notebook behind it is public. The gist below is the whole build, cell by cell, so you can run it yourself instead of taking the video's word for it.