{"id":47763448,"user":"kang","time":1776160541,"time_ago":"3 days ago","type":"comment","content":"

Doubt if you can make a dumb model smart by feeding it proofs","comments":[{"id":47764180,"user":"EagnaIonat","time":1776166079,"time_ago":"3 days ago","type":"comment","content":"

https://www.promptingguide.ai/techniques/knowledge","comments":[{"id":47765050,"user":"Tade0","time":1776171331,"time_ago":"3 days ago","type":"comment","content":"

Sohnds like a great way to fill up the context before you even start.","comments":[{"id":47765280,"user":"falcor84","time":1776172678,"time_ago":"3 days ago","type":"comment","content":"

Yes, what's your point? That is literally what it does - it adds relevant knowledge to the prompt before generating a response, in order to ground it me effectively.","comments":[{"id":47767675,"user":"Tade0","time":1776183768,"time_ago":"3 days ago","type":"comment","content":"

My point is that this doesn't scale. You want the LLM to have knowledge embedded in its weights, not prompted in.","comments":[{"id":47768330,"user":"EagnaIonat","time":1776186513,"time_ago":"3 days ago","type":"comment","content":"

It scales fine if done correctly.

Even with the weights the extra context allows it to move to the correct space.

Much the same as humans there are terms that are meaningless without knowing the context.","comments":[{"id":47771427,"user":"kang","time":1776200278,"time_ago":"3 days ago","type":"comment","content":"

Would it be possible to make GPT3 from GPT2 just by prompting? It doesn't work/scale","comments":[{"id":47774988,"user":"EagnaIonat","time":1776230480,"time_ago":"2 days ago","type":"comment","content":"

Bit of a straw-man there.","comments":[],"comments_count":0,"level":6,"url":"item?id=47774988"}],"comments_count":1,"level":5,"url":"item?id=47771427"}],"comments_count":2,"level":4,"url":"item?id=47768330"}],"comments_count":3,"level":3,"url":"item?id=47767675"}],"comments_count":4,"level":2,"url":"item?id=47765280"}],"comments_count":5,"level":1,"url":"item?id=47765050"}],"comments_count":6,"level":0,"url":"item?id=47764180"}],"comments_count":7,"url":"item?id=47763448"}

Comment by kang