← Back to context

Comment by peab

3 days ago

gemini flash is probably the best model for visual tasks right now. they also make it really easy to ingest videos

15 comments

peab

Reply
icelancer  3 days ago

Crazy it's still the only video understanding endpoint. It's what I use it for and no other model even offers a competitor.

  • mike_hearn  3 days ago

    You probably can't build such a model without unlimited access to YouTube and Google has been tightening the screws on that over the years pretty systematically.

  • peab  2 days ago

    to be fair, all it's doing is sampling the frames and maybe doing transcription, if I'm not mistaken. So you can do it with the other models too, you just need to sample the frames yourself and do the transcript yourself

    • icelancer  1 day ago

      it does this at a variable rate of frames which you can set - not sure if it is transcribing or natively understanding audio, but I think it's the latter since it is much faster than most transcription models I am aware of

      Regardless you are right - I can roll my own.... but why

pants2  3 days ago

Yes, was going to say I use it exclusively for video and audio. The ability to give it a YouTube link through the API and ask questions about it is awesome

ipsod  3 days ago

Also best at OpenSCAD, seemingly for the same reason, at least in terms of "iterate on a design, comparing visual output to target".

  • nl  3 days ago

    I've found Sol excellent at OpenSCAD.

    But I don't do "compare visual output to target" as much as 3D reasoning type tasks (eg: "Build a G1 curve where the -X face meets the +Z face" etc)

  • gunalx  3 days ago

    +1 gemini models where really the only ones fullt grasping spatial reasoning even compared to opus (at least when i last cared to check it)

  • sourweasel  3 days ago

    Are you manually rendering previews of its OpenSCAD output to create images for it to review, or do you have a workflow that automates that?

    • ipsod  3 days ago

      I've built some tools using AI. A custom GUI that has "copy context" and "copy image" buttons, to make prompting easy, and the camera position is persisted to disk on each change so that the agent can run a command to get a screenshot of it.

      I'm thinking about inlining an AI chat window directly - I guess I might fire off a prompt to do that right now.

    • gunalx  3 days ago

      Personally both, i use openscad with opencode, and paste images but it is pretty often the modell decides by itself it wants to see a render and uses the render shell commands to get a image to look at.

wxw  3 days ago

Ah, multimodal is a great point. I'll need to try that some time.