← Back to context

Comment by simonw

4 days ago

I use ffmpeg multiple times a week thanks to LLMs. It's my top use-case for my "llm cmd" tool:

  uv tool install llm
  llm install llm-cmd

  llm cmd use ffmpeg to extract audio from myfile.mov and save that as mp3

https://github.com/simonw/llm-cmd

73 comments

simonw

Reply
resonious  4 days ago

I tried this (though with a different tool called aichat) for extremely simple stuff like just "convert this mov to mp4" and it generated overly complex commands that failed due to missing libraries. When I removed the "crap" from the commands, they worked.

So much like code assistance, they still need a fair amount of baby sitting. A good boost for experienced operators but might suck for beginners.

  • cm2187  3 days ago

    Plus you need to know the format of your source file to design the command correctly. How many audio tracks, is the first video track a thumbnail or the video, are the subtitles tracks forced, etc.

    And in some situations ffmpeg has some warts you have to go around. Like they introduced recently a moronic change of behaviour where the first sub tracks becomes forced/default irrespective of the original forced/default flag of the source. You need to add "-default_mode infer_no_subs" to counter that.

    • Philpax  3 days ago

      I usually just paste the output of `ffprobe` into Claude when it's ambiguous. Works a treat.

  • Over2Chars  3 days ago

    My feelings exactly, but I think that's OK!

    It's another tool and one that might actually improve with time. I don't see GNU's man pages getting any better spontaneously.

    Whoa, what if they started to use AI to auto-generate man pages...

    • michaelcampbell  2 days ago

      > what if they started to use AI to auto-generate man pages...

      Then they'd be wrong about 20% of the time, and still no one would read them. ;-)

      (NB: I'm of the age that I do read them, but I think I'm in the minority.)

  • BiteCode_dev  3 days ago

    Reading this feels like seing a guy getting his first car in 1920 and complaining he still has to drive it himself.

    • imiric  3 days ago

      To me it's more like a guy getting his first car and complaining that the car is driving him in a direction that may or may not be correct, despite his best efforts to steer it where he wants to go. And the only way to know whether he ends up in the right place is to get out of the car, look around, and maybe ask more experienced drivers. Failing that, his only option is to get back in and hope to be luckier in the next trip.

      Or he can just ditch the car and walk. Sure, it's slower and requires more effort, but he knows exactly how to do that and where it will take him.

    • LeoWattenberg  3 days ago

      The beer brewers in my home town used to have a self-driving horse and cart which knew the daily delivery route going by all pubs and didn't really need a human to steer it or indeed be conscious during the trip. Expectedly, the delivery guy would get drunk first thing in the morning and just get carted about collecting the money.

    • pbhjpbhj  3 days ago

      Pony & trap could be largely self-driving, after an initial training period. That would have been a distinct negative to "upgrading" for some, I'd imagine.

      1 reply →

    • atoav  3 days ago

      Sell that guy probably got carried home by his horse after drinking half a bottle of whiskey, so maybe he had a point.

    • shriek  3 days ago

      Or maybe calling a cab and telling the cab driver each direction to get to the destination instead of the cab driver just taking you there.

  • assimpleaspossi  3 days ago

    My experience exactly.

    I no longer check with these AI tools after a number of attempts. Unrelated, a friend thought there was a NFL football game last Saturday at noon. Checking with Google's Gemini, it said "no", but there was one between two teams whose season had ended two weeks before at 1:00 Eastern Time and 2:00 Central. (The times are backwards.)

  • sdesol  3 days ago

    > "convert this mov to mp4"

    Did any of the commands look like the ones in the left window:

    https://beta.gitsense.com/?chats=12850fe4-ffb1-4618-9215-c13...

    The left window contains a summary of all the LLMs asked, including all commands. The right window contains the individual LLM responses.

    I asked about gotchas with missing libraries as well, and Sonnet 3.5 said there were. Were these the same libraries that were missing for you?

  • keeganpoppen  3 days ago

    what exactly do you want the llm to do here? if the ask was so unambiguous and simple that it could be reliably generated, then the interface wouldn't be so complicated to use in the first place! LLMs are not in any way best suited for one-shot prompt => perfect output, and expectations to that effect are extremely unreasonable. the reason why LLMs are still hard for beginners to use is because the software is hard to use correctly. as with LLM output goes life itself: the results you get from using a tool can only ever be as good as the (mental) model used to choose that tool & the inputs to begin with. if all the information required to generate the output were contained by the initial prompt, then there would be absolutely no need to use the LLM at all in the first place.

  • Philpax  3 days ago

    Hate to be that guy, but which LLM was doing the generation? GPT-4 Turbo / Claude 3.x have not really let me down in generating ffmpeg commands - especially for basic requests - with most of their failures resulting from domain-specific vagaries that an expert would need to weigh in on m

pmarreck  3 days ago

A while back I simply wrote my own bash function for this called `please`

as in

    bash> please "use ffmpeg to extract audio from myfile.mov and save it as mp3"

It will then courteously show you the command it wants to run before you agree to do it.

Here is the whole thing, with its two dependent functions, so that people stop writing their own versions of this lol. All it needs is an OPENAI_API_KEY, feel free to modify for other LLMs

EDIT: Moved to a gist: https://gist.github.com/pmarreck/9ce17f7996347dd532f3e20a2a3...

Suggestions welcome- for example I want to add a feature that either just copies it (for further modification) or prepopulates the command line with it somehow (possibly for further modification, or even for skipping the approval step)

atoav  3 days ago

Did you just invent the LLM-equivalent of curl-piping unread shell scripts into sh?

I am sure that will never cause any problems.

dekhn  4 days ago

"The future is already here. It's just not very well distributed"

(honestly, the work you share is very inspiring)

zahlman  4 days ago

>This will then be displayed in your terminal ready for you to edit it, or hit <enter> to execute the prompt. If the command doesnt't look right, hit Ctrl+C to cancel.

I appreciate the UI choice here. I have yet to do anything with AI (consciously and deliberately, anyway) but this sort of thing is exactly what I imagine as a proper use case.

  • hnuser123456  3 days ago

    Just like all other code. There will be user-respecting open source code and tools, and there's user-disrespecting profitable closed code that makes too many decisions for you.

th0ma5  3 days ago

You should figure out what went wrong for the other commenter and fix your tool.

mrweasel  3 days ago

While I love that that works, I still feel like just maybe ffmpeg needs a better interface. Not necessarily a GUI, just a better designed command line.

Waterluvian  3 days ago

I think I’m finally sold on actually attempting to add some LLM to my toolbelt.

As a helper and not a replacement, this sounds grand. Like the most epic autocomplete. Because I hate how much time I waste trying to figure out the command line incantation when I already know precisely what I want to do. It’s the weakest part of the command line experience.

  • Over2Chars  3 days ago

    But possibly the most rewarding. The struggle is its own reward that pays off later many times over.

    • hbn  3 days ago

      There are times I feel minor guilt for using an LLM to relieve brainwork, like figuring out an algorithm. That's probably a skill I should continue practicing for my own sake.

      ffmpeg commands though? It's really not a practical skill outside of using ffmpeg. There's nothing really rewarding to me about memorizing awkwardly designed CLI incantations. It's all arbitrary.

      1 reply →