Comment by throwawayboise
5 years ago
About 30 years ago one of my managers used to say "get the data model right and the application writes itself" and I have found that to be mostly true. What I have also often found is that people who create spreadsheets in business don't understand data modeling and even if the spreadsheet solves some business problem it's often very brittle and hard to change and adapt or generalize.
The spreadsheet structure point is an interesting challenge - I think often a spreadsheet ends up as the de facto model of a process, but often with, as you say, some redundancy, excessive flattening, and other structural issues that can make it more diffcult to build an app around.
The nice thing, though, is that shifting this structure around does not mean changing the process being modelled - it's more just a necessary part of making a more powerful tool to support it.
It's as you say, since the process is known, it's usually very clear exactly how the app should be, which under our model can inform how to shift the structure of the spreadsheet accordingly in pretty practical way. It's cool to see the same thing work in both directions!
From my experience working with some business-side using spreadsheets: yes, usually spreadsheets end as the de facto model of a process but not necessarily an efficient model or an easily replicable one.
In banks I know of some long-living spreadsheets that have been patched so much that it takes a real human weeks to months of work to disentangle the mess of macros and recalculations onto a streamlined script/process. Sometimes the resulting model diverges in quirky ways that are manually patched, I've seen diversions due to datetime issues (time zones, summer time, leap days, incorrect assumptions about dates, etc.) that were noted by side-effects of side-effects and the manual patching didn't help at all to debug the root cause.
I think that spreasheets are incredibly powerful, but the main reason for that power is that they are quite free-flowing and that invites the human creativity to solve problems with the limited set of tools and knowledge some users have, and some of those are in quite high technical positions using spreadsheets daily for years.
I believe you might have a killer product but I had so many headaches with spreadsheets that I wouldn't like to be working in that space.
My first project out of college was working on an internal metrics tool for a company. Their prior one was basically Excel; a guy who was due to retire had, back in the 90s, written an entire DSL with VBA, that could load files, run various functions on them, and output graphs.
Thing is, no one except him knew the DSL; everyone in the company relied on it, but they relied on him to write the DSL to compile their inputs into the output they wanted.
The rewrite included an actual database, proper data ingestion, and a nice clean frontend. The methods of aggregating data were reduced and standardized, as were the types of simulations and metrics that could be reported on; the flexibility was drastically reduced. However, the practical usage for everyone was drastically increased, because it moved from "we can't do anything different unless we get (expensive due to retire person's time)" to "we can do it ourselves.
I'm very jaded toward no/low code, in general, and that experience is partly the reason why. There isn't a sweet spot, that I've seen, that allows for non-technical people to have the control they want. And that was true even with spreadsheets.
The less nice thing, though, is that the model of the process you're starting from -- the actual spreadsheet -- has, as you say, these structural problems. And since (some speculation here, but I very much suspect that) many different processes, after having been so mangled, will end up in the same redundant, excessively-flattened structure, you can't determine from the spreadsheet alone which of these different processes it is supposed to encapsulate.
So before you can start "shifting this structure around" you'll still have to go through a standard business analysis process to find out what you are going to shift it into. And if you're already doing that... Well, then most of your promise of automation is out the window already, so what's the use of having the actual implementation done in some weird newfangled "no-code" or "low-code" tool?
This!
Understand the data and the application is simple and easy to understand. Start with a flashy GUI and your data is a mess.