Comment by TySchultz
2 years ago
I had nearly the same idea as you around surfacing what is "important" vs just a large list of RSS articles.
The main differences compared to what you are thinking are two things. One for the `Significance of the event` I've used the number of publishers talking about that event. So more publishers == more important. Two, I've done this in a daily fashion instead of a weekly report.
I can also confirm that the LM has the capability to do at least a days worth (2500+) of articles. I would doubt its capability to produce an entire article but it does a great job at a small summary.
Here is the link if you wanted to check it out. https://apps.apple.com/us/app/quill-news-digest/id1669557131
Are you curating the list of publishers somehow? I would imagine the AP/AFP newswire repost circuit/echo chamber would result in overestimating importance of a lot of crapola and underestimate the importance of investigative pieces for example.
This was an issue when I first started. With minimal sources a lot of time the top collections were low quality SEO articles.
After adding a sufficient amount of sources I've noticed a decent reduction in the echo chamber. Although by ranking importance by the most talked about topic, it is going to have some sort of echo chamber.
Adding left and right leaning publishers for instance has helped. Although one might say something is good and something is bad, the embeddings pick it up as the same topic.
In a way it also cuts through bias.
I just downloaded it, looks pretty useful especially for news sections I don’t follow particularly closely.
Will check this out over a few days.
Out of curiosity have you looked at deep you can go while still getting quality results (i.e. beyond top 2-3 in each heading)?