Comment by droidjj
13 hours ago
I’m sure there are a wide variety of experiences out there, but here’s my perspective as a former biglaw associate and current solo litigator:
I have had some success using frontier models from the last 6ish months, but only when I can break up my work into discrete and verifiable tasks. For example, I had ~15k pages of discovery I needed to dig through for a summary judgment motion. Instead of just asking Claude to find the best evidence, I asked it first to run a clean, high quality OCR pass (it was almost entirely PDFs). Then I had it generate embeddings and write some reusable python scripts to make keyword and semantic searching easy for agents. While I was writing the brief, I would routinely ask my agent (Claude Code) to use both keyword and semantic searching to find the best evidence supporting whatever assertion I was trying to make. I trusted it because there were traces I could follow.
In other cases/situations, I’ve tried just giving a model access to all the docs and saying “write a brief arguing X,” but it’s always terrible at this. It writes briefs with lots of evocative jargon and rhetorical flourish, but a low signal-to-noise ratio.
Again, I’m sure others’ experiences differ based on workflow, legal area, etc.
Agreed. Six months ago, it was basically a gloried grammarly.
But lately, I’ve been taking hints from the “company brain” models, where it develops a running model of the case, and assesses each new piece as it comes in and updates the file.
I’ve also been using “Ralph Wiggum”-type models where you pass letter or contract drafts back and forth between agents with different goals (rules compliance, grammar, conciseness, ai slop detector, an opposing counsel critic, etc.). After a few rounds, it’s not perfect — but I start with a very good first draft in my hands.
What are you using for semantic searching?