Comment by amelius

16 hours ago

Speaking how which, how are economists using AI? Are they getting better at making predictions?

Essentially all of economic theory is aimed at explaining, not predicting. The distinction between the two goals [0] is sometimes under-appreciated within the profession, and almost always under-appreciated outside of it.

Most predictive tools in economics and finance have “surprisingly” little economic content; but once you understand the distinction between the two goals, it should be unsurprising that predictive models tend to make few economic assumptions, relying rather on general statistical techiniques or on econometrics that incorporate a minimum of theory [1]. From that understanding comes the humbling realization that predicting the future is quite difficult in a context in which the relevant processes are continually seeking an equilibrium that often implies unpredictability. [2]

I’m not an economist, but I do a lot of applied financial-economic modeling. State-of-the-art LLMs are really, really terrible at economic intuition. They will hinder, not help, in formulating an economic model, which is a process of coming up with a set of modeling assumptions that lead to a useful (implicitly, tractable) model. LLMs are, however, quite good at math, and I’ve found them very useful in iterating through different sets of modeling assumptions to identify those that lead somewhere useful. Not having to work out all of the mathematical details myself, and thereby avoiding getting lost in the weeds and being better able to maintain a higher-level perspective on what I’m trying to accomplish, has accelerated my work immensely. But it’s a process of leading the LLM by the nose the whole time and asking it to fill in the details.

I should note, thought, that if you indotend “AI” to mean more than LLMs, them yes, there is starting to be a lot of good work done on predictive economic models that use specialized neural networks as black-box functions to compute model quantities that are otherwise difficult to come up with, just as is also happening in applied physics and other fields.

0. https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf

1. Many explanatory economic models refer to quantities that are fundamentally or practically unobservable or unidentifiable. Much of economics is built on models that were designed to provide a formal, logical basis for understanding the economic world, which is often quite unintuitive. (For example, many intelligent people uneducated in economics exhibit intuitions opposite of basic economic ideas like opportunity cost or comparative advantage.) Models of this sort have been very influential in determining the trajectory of economic thought, but they are often effectively impossible to calibrate to the real world.

2. The most influential and effective economic ideas fall into a third class: ideas that have created their own reality by shaping the way people think in a way that gives rise to the results the models explain or predict. This phenomenon is most evident in finance, where ideas like the various forms of the efficient market hypothesis, the CAPM, and the Black–Scholes model and its follow-one have arguably provided a framework that has reshaped the ways financial practitioners behave to such an extent that financial markets now conform much more closely to what the models describe than was formerly the case. Donald MacKenzie’s book An Engine, Not a Camera is an excellent study of this phenomenon: https://mitpress.mit.edu/9780262633673/an-engine-not-a-camer...

  • >Much of economics is built on models that were designed to provide a formal, logical basis for understanding the economic world, which is often quite unintuitive.

    I disagree. It's purposefully unintuitive.

    >(For example, many intelligent people uneducated in economics exhibit intuitions opposite of basic economic ideas like opportunity cost or comparative advantage.)

    Most people don't believe in comparative advantage. They believe in something that economists can explain away as comparative advantage.

    All unconsumed fixed size investments will result in something that is mathematically the same as comparative advantage. This is the intuitive view that people have. You go to university and get a 5 year degree. Now your cost basis for work that suits your expertise is much lower than for work that is out of expertise. A worker buys an expensive machine, now the cost basis for hiring the guy with the machine is lower than buying your own machine.

    This also explains why specialization emerges: All specialization is basically a form of an investment that has some residual left over results that can be monetized in the future. If there was no residual it would be as if you forgot your education and at that point the investment is fully consumed and you turn back into a non-specialized worker.

    All of this is incredibly intuitive, but economists instead insist on an invisible "factor" [0] to drive efficient production.

    [0] The "factor" concept implies comparative advantage exists first rather than emerges as a result of past decisions.

    • Not sure comparative advantage needs past decisions to happen - could just be randomly distributed resources, for example.

That would require AI to be accurate

  • It wouldn't require perfect accuracy, just rough accuracy and a human to confirm, and it's already more than good enough for that. I do not understand this confusion surrounding modern math.

Here's the thing about economists... The loudest ones don't want to be correct, they want to be influential. The ones who can actually make good predictions work for banks and hedge funds lol.

Honestly, it's a bit of a disappointment

- Many more mediocre papers written (mediocre ideas, implementation, claude-isms everywhere)

- Much easier to try every possible combination of a regression in order to show the result you want (same for theorists).

The one thing I'm happy about is it's now much easier to extract historical data from old documents from Google Books. Still not perfect, but takes you 95% there. And creating plots and datavis just for quick exploration is super fast.