Comment by Animats

6 days ago

Here are two routine problems I have to solve at the moment. Can any of the current LLM systems do either?

1. Input is an 256x256 pixel elevation map stored as a greyscale .png file, and a minimum and maximum elevation. A pixel value of 0 corresponds to the minimum elevation, and a pixel value of 255 corresponds to the maximum elevation. Read in the .png file and the elevation limits. Then construct a 256x256 floating point array of heights. From that array, construct a triangle mesh with X and Y dimensions 0..255. Perform a mesh reduction operation on the triangle mesh to reduce the number of triangles. Mesh reduction must not generate holes in the mesh. From the reduced mesh, generate a glTF file where the UV parameters run from 0.0 to 1.0 along the X and Y axes.

2. Given four glTF files constructed as above, corresponding to four quadrants of a larger square, construct a single 511x511 mesh which combines all four input meshes to cover a larger area. Because the input meshes are 0..255, not 0..256, there will be gaps where the four quadrants meet. Fill those gaps with reasonable triangles. Perform a mesh reduction as above. From the reduced mesh, generate a glTF file where the UV parameters run from 0.0 to 1.0 along the X and Y axes.

Rust code is preferred; Python code is acceptable.

So, what service should I sign up for?

14 comments

Animats

Reply
csomar  6 days ago

Tackling 1: No current LLM will be able to do that. Create your Rust project, figure out the dependencies that could be useful on your own, then start by asking which types can be expressed. Then after you have revised the types, figure out (with or without LLM) which functions have to be implemented. Then ask LLMs to implement these functions with tests. With some modifications, you should be able to have a final working solution.

Edit: took a while but here is the code generated by DeepSeek: https://gist.github.com/omarabid/b30195011abf570fde2a084cd11... If you have an example file I could give it a test run.

  • Animats  6 days ago

    > Tackling 1: No current LLM will be able to do that.

    Github Copilot seems to be able to do it. Mostly because it knows how to call Meshlab. The documentation for Meshlab is hard to search, and having an LLM find the correct function is itself helpful.

    Also, it looks like quadric decimation has improved since the last time I looked at it. That algorithm, if pushed too hard, will generate meshes with holes. Now there are options ("preservetopology") which supposedly prevent that. I last looked at that a few years ago, and the only mesh decimator which handled that correctly was the one inside Unreal Engine Editor.

    Now I have to test all this stuff, of course.

  • energy123  6 days ago

    How have you found LLMs' Rust capabilities compared to Python or Typescript or other more common languages with more publicly available data?

    • Animats  6 days ago

      I tried some LLMs about six months ago, and got a halfway decent translation between Linden Scripting Language and Lua. I didn't expect that to work at all, but it did, despite very different syntax. The parent article says that if you haven't tried this in six months, it's time to try again. It looks like it's going to be useful for game geometry wrangling, at least.

      LLMs seem to be good at finding obscure library functions from a description of the function needed, something where where keyword searches aren't too helpful. Previously, searching Stack Overflow helped, but nobody is posting there any more. Good reference manuals are almost historical artifacts now.

    • csomar  6 days ago

      Rust is the best if you carefully create and document your types. Sometimes you’ll get stuck in lifetime issue but it’s a sign the LLM didn’t do a good job.

JKCalhoun  6 days ago

Why don't you paste that into Claude, ChatGPT — report back what you find?

  • Animats  6 days ago

    I tried Github Copilot, free mode, on #1. The Python code outsourced all the hard work to numpy and pymeshlab, which is fine. Copilot wrote code to generate triangles. It's a reasonable job. Copilot's knowledge of what to call is better than mine. I have to try it now, but it did OK.

    On #2, it set up the problem, but bailed on the gap filling part with

       # (This is a complex step; see note at bottom.)

    That's not surprising, since I didn't tell it how to solve the problem. Can any of the premium systems do #2?

    • nickpsecurity  6 days ago

      When I used it, Copilot was for doing autocomplete on little, routine procedures. Like a few operations done on a list. I used it to fill in the blanks of tedious stuff in what I was already coding myself. It worked well at that.

      However, if it took creativity or real analysis, I had to throw the big models at it with a carefully-written prompt. You want to use their recent, best, big models. I used to include precise descriptions of data structures and function API's, too, which helped a lot. From there, tweak the description until you find pieces that get the result.

      Another thing. It will eventually synthesize code that's close but not quite it. I found that asking it to make small changes, one at a time, would help for a while. Ex: "Modify that code to do X" or "...X in part Y." It eventually started hallucinating in loops. I always had to fix some amount of what it generated but it still saved time.

      I hope that helps.

agnishom  5 days ago

This is a great example.

Supposedly, you know something about this problem. If you use an LLM to craft a solution, that's probably fine since you'd know how well you did.

I know nothing about this problem. I could use an LLM to delude myself into thinking that I do. Should my boss be very happy about that? They probably shouldn't