Comment by g9550684
10 hours ago
these are obvious techniques from array world, j/k/apl, and the source of e.g. arthur's bold speed claims.
personally I've discovered it insufficient to "just" convert tables to lists, you also need to understand the rest of the array principles to then effectively manipulate your data, and just to be able to hold compute in your head. because I believe in this approach I spent time learning j and k, but the end result is that my solutions become too alien for the general practitioner. it becomes apl written in whatever host language.
this is something that is not addressed in a lot of DOD talks, what happens when you do the full realization of technique: bulk list primitives, bulk transforms, SIMD optimizations, list compression, etc. and it's also the reason people claim this only works in narrow scopes (like gamedev). in reality you can write all your code for lack of better term the apl way, you're just going to make it unreadable to non apl practitioners. I'm not quite sure how to reconcile this in general, short of forcing apl to be part of general CS curriculum.
> convert tables to lists
array languages give you concepts for thinking about these problems where you can succinctly express that entire talk in a single sentence, something like "inverted tables improve cache locality and optimize for time and space, prefer them where it matters". yes, got it, also a well known conclusion in array world.
a table is a 2 dimensional data that stores your records row by row and the items of a column all have the same data type. the way an array of structs would. an inverted table is a list of original table's columns. like a struct of arrays.
so if you have a table,
there's an idiom for converting it to an inverted table
you can then splice it across variables,
so you have converted a table to lists.