Comment by kirubakaran

4 years ago

I extract the title, headings (h1,h2,h3), and some meta data from the page content and send that along with the prompt to gpt-3.5 to pick the relevant tags from a set of tags.

So that is how you deal with the length limit? Did you just make up a list of tags?

  • Yes, I played around with sending first n chars from the web page text etc, but found that sending headings is to pick the tags.

    I extracted the list from here as the starting point: https://lobste.rs/tags I spend a lot of time on HN haha, so I was able to expand on that list and I think the current list is pretty comprehensive. I can share the full list if you're interested.