Comment by minimaxir

3 months ago

I have been doing more experiments with what I have now been calling agentic iterative optimization: telling the LLM to optimize code such that it speeds up all real-world-representative benchmarks by X% without cheating or causing regressions in both tests and performance metrics (e.g. MSE for statistical algorithms or file size in the case of something such as image compression). This is done using Rust where there are more low-level levers to tweak for performance than something like Python.

Opus 4.6/4.7 was consistently successful at getting 2-3x speed improvement with just one pass. It can also do the inverse: improve the performance metrics for better quality without causing a significant regression in speed. Then GPT-5.5 turned out to be much better at this workflow, often getting a multiplicative 1.5x-2x improvement above what Opus could do.

I now have quite a few GPT-5.5-optimized projects in various domains that are feature complete and are substantially more performant than existing SOTA implementations that I plan to open source as soon as possible: the bottleneck is polish as usual.

6 comments

minimaxir

Reply
csutil-com  3 months ago

Very interesting, could you share they prompts you typically use for this?

Something like this?

You are an Elite Performance Engineer and Autonomous Optimization Agent. Your primary goal is to iteratively optimize the provided codebase to maximize execution speed and efficiency (e.g., reduce CPU cycles, memory allocation, or network latency) WITHOUT altering the external behavior or causing any test regressions.

### CORE DIRECTIVES 1. METRIC-DRIVEN: You will be provided with benchmark results, profiler logs, or execution times. Your only measure of success is a statistically significant improvement in these metrics. 2. ZERO REGRESSION: The test suite MUST pass 100%. If a test fails after your modification, your immediate next step is to diagnose the failure and either fix the logic or revert to the last working state. 3. NO CHEATING: Do not "hardcode" solutions to bypass the specific benchmark inputs. The optimization must be generalized and algorithmically sound for all valid inputs. 4. ISOLATED CHANGES: Make precise, localized changes. Do not refactor architecture unless absolutely necessary for the performance gain.

### THE ITERATION LOOP When instructed to optimize, follow this thought process strictly using <thought> tags before writing any code: - ANALYZE: Review the current code and the latest benchmark/profiler feedback. Identify the specific bottleneck (e.g., redundant loops, excessive object creation, DOM reflows, synchronous blocking). - HYPOTHESIZE: Formulate exactly ONE hypothesis for improvement (e.g., "Replacing the array filter+map chain with a single reduce pass will save N allocations"). - IMPLEMENT: Output the precise code modifications required for the hypothesis. - EVALUATE (Mental Check): Ask yourself if this change introduces edge-case bugs (e.g., handling of nulls, empty arrays, async state).

If a previous optimization attempt resulted in a slower benchmark or a failed test, explicitly state WHY it failed in your thoughts before attempting a different approach.

Proceed with your first analysis of the provided files and await the baseline benchmark metrics.

  • minimaxir  3 months ago

    This is the current version of my prompt, which is tagged in a Markdown file to "implement correctly and comprehensively". The second paragraph is a recent addition that unlocked further speed improvements after I thought my repos had already converged. This prompt assumes benchmarks are already present in the repo.

        Optimize the performance of this Rust/Python X crate as much as possible without causing ANY regressions.

    This is a very difficult problem and traditional statistical approaches **WILL** fail to hit the specified metric constraint. You have permission and encouragement to investigate more radical fundamental low-level changes to hit the desired metrics. You have permission and encouragement to invent completely new statistical/machine learning algorithms that have never been before been utilized for this problem.

   First, **before making any changes**, run the Rust benchmarks and Python benchmarks to establish a True Performance Baseline for both speed and metric performance. Return the absolute and relative results to the True Performance Baseline to the user as a Markdown table.

   Then, optimize the crate code such that ensure that ALL Python/Rust benchmarks are **atleast 1.2x faster** from the True Performance Baseline; ideally as fast as possible. You are only allowed **up to a 5% metric regression (e.g. accuracy)** to accomplish this. NEVER hack the benchmarks to accomplish this reduction, only iterate on the library code.

   Do not import similar implementations from other Rust crates: you MUST implement from scratch.

   You may use ANY techniques to do so (e.g. import new crates) other than adding `unsafe` code. **REPEAT THIS PROCESS UNTIL BENCHMARK PERFORMANCE CONVERGES AND YOU ARE OUT OF OPTIMIZATION IDEAS.** You have permission to keep iterating. After each benchmark iteration, return the absolute and relative results to the True Performance Baseline to the user as a Markdown table.

   Prioritize making quick/high-impact wins iteratively and making changes accordingly. Do not overthink the necessary changes.

    I am also aware of the flaws in the prompt but if it works it works. AGENTS.md has other quality constraints.

  • thrw045  3 months ago

    Depending on how complex the code is, you don't need that big prompt with ChatGPT.

    I have sped up a project by simply saying "What are all the possible ways I can speed up this code?" Then it'll list everything it finds, then ask it to rewrite the code.

    Edit: Also, I find I didn't need to do this (because a speed up implies semantic similarity), but you can also add "change it without altering the semantics of the code" and in this way it'll be the same and should pass tests