Comment by AbstractH24
4 months ago
What are you using to build the dictionary? Particularly when it encounters something you've never seen before.
This is really interesting to me.
4 months ago
What are you using to build the dictionary? Particularly when it encounters something you've never seen before.
This is really interesting to me.
Continuing the brands example, by default I store all of the brands returned as is (in SQL). On occasion, I will manually come across different variations of a brand that I decide is better combined into a primary brand. All of the secondary brands get marked as relating to a primary brand. Then the next time a new ad gets tagged as a secondary brand, I know I can use the primary brand instead.
So in essence, the process is what I might call 'eventually modelled' (to borrow from the concept of eventual consistency). I use the LLM entities as is, and gradually conform them to my desired ontology as I discover the correct ontology over time.