Comment by collimarco
8 hours ago
What about clean, semantic HTML?
It was already optimized for bots and search engines (which are bots) and it has been used for decades. Why we need to serve in markdown now?
There are also many parts of the HTML, like navs, that are useful for bots and AI and may be removed in the markdown version.
>What about clean, semantic HTML?
Which React package is this?
I agree. I think a lot of people here are assuming that the full HTML retrieved has to go into the LLM eating up tokens. But why wouldn't the agent try to clean up first and remove bloat and convert to markdown itself, before feeding into LLM. Semantic HTML would make that easier.
> But why wouldn't the agent try to clean up first and remove bloat and convert to markdown itself, before feeding into LLM.
There's no "agent". It's a few wrappers around API calls in a trenchcoat.
Presumably markdown uses far fewer tokens.
Markdown isn't as expressive. Not all HTML content can be converted to Markdown without losing some of the semantics
Is that even true? I most often use HTML. HTML is about 5%-20% more tokens than a similar Markdown. As a rule of thumb, the number of tags/structural tokens doubles, when going from markdown to html, while the rest don't change much. On the other hand, I can view HTML without any extra/unusual tools. And composing HTML when I need a bit of structure is far easier than composing markdown.
> On the other hand, I can view HTML without any extra/unusual tools. And composing HTML when I need a bit of structure is far easier than composing markdown.
This is kind of the opposite of reality no? Markdown is just plain text and meant to be human readable. You don't need XML tags to read and write it, opposed to html where you do and you need a browser to properly view it.
1 reply →
That's highly dependent on what sites you're visiting. Take a look around at a lot of modern sites, there's a sea of divs and spans. Markdown conversion helps LLMs a lot.
1 reply →
That used to matter to me back in the days when the best models still only accepted ~32,000 tokens, but these days even the models that run on my laptop are happy with ~100,000 and the hosted models I use take ~200,000 or more.
They accept more tokens these days, but they are still more accurate with a shorter context [1].
[1] https://arxiv.org/abs/2307.03172
If it's one of many tool calls, I'd assume that less is more.