Comment by gspetr

> Combine A + B

Really, compress. Compression is a useful proxy for intelligence.

To compress data efficiently, an entity must discover the underlying patterns, structures, and regularities within that data. Identifying these structures is precisely what we define as "understanding" or "intelligence.

Anything we put into writing can be thought of as a string of data. The Kolmogorov complexity of a string of data is the length of the shortest possible computer program that can output that string.

A good compressor doesn't just memorize (yes, there's a reason LLM writing features the "it's not just X, it's Y pattern"), it builds a generalized model to handle unseen, similar data. LLMs are trained by compressing the internet into weights, they use this to write new code.

When an LLM is trained, it compresses ...bytes of human text into a few ...bytes of neural network weights. When you ask it a question, it uses that compressed model to generate (decompress) a coherent response.

Consider the dichotomy:

LLMs display emergent reasoning, coding capabilities, and logic because they were forced to compress human language so tightly.

LLMs often suffer from "hallucinations" and struggle with novel logic puzzles because compression encourages mimicking statistical patterns rather than building a true causal model of the physical world.

An algorithm can achieve massive compression if it utilizes a highly complex, slow, or massive decompression engine. If a system takes 10 000 years to decompress a file, or requires an infinite amount of computing power (like Kolmogorov complexity), it cannot function as a practical intelligence. Real intelligence must operate within tight time and energy constraints.

> Maybe there's true brilliance out there

Brilliance is a trope. pg's 2009 "The Anatomy of Determination" deals with it: https://news.ycombinator.com/item?id=49309549

So what's left for mere mortals? Apparently, nourishing your ambition:

"So here in sum is how determination seems to work: it consists of willfulness balanced with discipline, aimed by ambition. And fortunately at least two of these three qualities can be cultivated. You may be able to increase your strength of will somewhat; you can definitely learn self-discipline; and almost everyone is practically malnourished when it comes to ambition."