Comment by melvinroest

9 hours ago

I've been doing something less formal. I stumbled upon Riaan Zoetmulder's free course on deep learning and medical image analysis [1] and found his article on spec-driven development [2]. He adapts the V-Model by specifying three things upfront: requirements, system design and architecture. The rest gets generated. He mentioned a study where they show that LLM assistance slowed down experienced open source devs on large codebases. The model doesn't know the implicit context. And to me that's the thing! An LLM should have an index of some sort.

So I vibe coded my own static analysis program where I just track my own function calls. It outputs a call graph of all my self-defined functions and shows the name (and Python type hints) of what it is calling (excluding standard library function, also only self-defined stuff). Running that program and sending the diff from time to time seems to have helped a lot already.

[1] https://www.riaanzoetmulder.com/courses/deep-learning-medica...

[2] https://www.riaanzoetmulder.com/articles/ai-assisted-program...

2 comments

melvinroest

Reply
xcubic  7 hours ago

I am not following, can you give a concrete example of your workflow?

  • melvinroest  4 hours ago

    In my agent file I explain that I have a static analyzer which generates a callgraph. On starting the agent runs ~/.agent/tools/__callgraph__/generate_callgraph.py

    It then gets to see callgraph.current.md and upon subsequent sessions callgraph.diff.md.

    Here is an example of some output that I currently have in callgraph.current.md

      ## src/components/Header.tsx

  - **export Header({ ... }: Props)** (start 9, end 54) → `useAuth`

  ## src/components/HelpTooltip.tsx

  - **export HelpTooltip({ ... }: Props)** (start 15, end 42) → (none)

  ## src/components/ResultsTable.tsx

  - **getHeaderLabel(col: { id: string; columnDef: { header?: unknown } }): string** (start 45, end 51) → (none)
  - **getCellValue(colId: string, original: KeywordResult): string** (start 53, end 74) → (none)
  - **export ResultsTable({ ... }: Props)** (start 76, end 406) → `getCellValue`

  ## src/components/SettingsDrawer.tsx

  - **export SettingsDrawer({ ... }: Props)** (start 158, end 336) → (none)

    For example:

    ResultsTable calls getCellValue.

    In these cases it's just one function but you also have stuff like

      **export Dashboard()** (start 36, end 635) → `getTopKeywords`, `normalizeText`, `searchKeywords`, `searchKeywordsMulti`, `searchSemantic`, `searchSemanticMulti`

    For the Python version it also gives the parameters and types along with it. I think the next thing I'd need to do is give self-defined type definitions. Doing things this way allows an LLM to not read all that much but to be able to reason relatively well over what the code does. The caveat is that you abstracted your code well. If you didn't, the LLM doesn't know your implementation.

    I probably should also add return types.