Comment by sixhobbits

9 hours ago

I do some similar charting etc with telegram data dumps that you can still get from the "telegram lite" desktop app even though they removed the export functionality from the main app.

For removing noise you might want to look into TF-IDF instead of the manual method described in the post that I didn't understand. It basically looks for words common across the whole corpus as noise or ones that appear within a specific chat much higher than the whole dataset as interesting.

You can also do some fun stuff by finding phrases used asymmetrically eg more by one person in the convo than the other, or over time.

Wordclouds per person are also fun!

6 comments

sixhobbits

Reply
valzevul  2 hours ago

Hi, OP here!

TF-IDF was the first thing I tried - it works great for stopwords but it doesn't handle cross-language bleed of filler words well, and the short life-event messages ("he died", etc) use common words and get aggressively down-weighted.

I had some asymmetry analysis when looking at directional sentiment and per-person question rates - that's fun indeed!

I also went with the Jaccard convergence and the endearment categories instead of wordclouds, so that I could see how word choices are changing across time.

sunaookami  7 hours ago

Huh, the export function in Telegram Desktop (https://github.com/telegramdesktop/tdesktop/) is still there. Click on the three dots while in a chat and then Export.