Comment by donatj
6 years ago
I was lead developer for a company ten years ago who developed tools for industrial companies. We had a steel rack manufacturer approach us with a panic inducing spreadsheet of the calculations they needed. It was 8 very full worksheets and included everything you could think of, including seismic data by zip code.
To build this proper was going to take our small team at least a couple months. The sales people didn’t like this. I had an idea however to extend a SpreadsheetML parser I’d written a few years prior with a formula evaluator. I had a prototype working within a couple days, I reimplemented a large number of Excel functions warts and all, and since Excel saved all the last computed values along side the formulas it was easy to automatically test. It was the first project that really taught me the value of unit testing.
It was a fun project and took around a month in the end, which was still over the amount of time sales had given us. The client actually sent us a fair number of revisions to the spreadsheet such that had we built it out any other way would have been a major project, but was as easy as swapping the spreadsheet the tool read. I left the company a few months later but I was sad to hear from coworkers they never did anything else with the tool. I thought it was a really neat hack.
Damn It. Where are these problems today? Or is it a matter of just choosing between getting paid vs passion? As a hirer in a FANG, I feel ashamed at how much we put some of the smartest and hard-working people to work on yet-another-crud-rest-api.
All the engineering companies that aren't software engineering companies. I work for a largish civil engineering and urban planing consultancy and we deal with all kinds of hard and interesting technical problems like this all the time.
The main downside is that the software development culture in these companies are at least a decade behind the state of the art and you will have to deal with weird things like TFS 2012 and no one having heard of the phrase "Unit Test". On the upside since there is no software development culture in place, there is less argument when you tell them you're moving all your code to gitlab and want to use F#.
I work at one of those rare manufacturing companies here in the bay area(pay isn't anywhere close to what FAANG median is).We tend to use Excel for everything from manufacturing quality Pareto charts, SPC charts and its a Big mess.All I hear from Big tech enterprise tech is IoT and datalakes but no immediate solution that aggregates all this data in manner that can be consumed by various LOB Apps that can do things with the data. So I am in the process of building one right now. But You are right. FAANG along with every other unicorn is running after the NBT, wherein there is so much more to automate in well established Non-big tech(Dino tech) like P&G case below.
You can do nearly anything in Excel/Office with VBA, from automating manual processes, to BI reporting, to gluing together any and all data sources, it's just that CS type geeks hate it and refuse to learn it.
Being limited to only Office for a few months has not made me think "how horrible it is not to have real tools", but rather "how amazing is the amount of money and effort people waste on 'enterprise solutions' that are both orders of magnitude more expensive and inferior".
6 replies →
Damn It. Where are these problems today?
They're still here. When we had to generate reports for municipalities, and ingest new data from non-tech sorts of people. What did we choose? Excel 2003 XML. Because it's what we could import easily.
I feel ashamed at how much we put some of the smartest and hard-working people to work on yet-another-crud-rest-api.
Why? Excel is inherently poorly specified and the file format support matrix is a bit… unwieldly. Interoperability is terrible, any sort of multi-user use case becomes an exercise in archaic procedures, and the various scripting languages, while powerful, are janky as fuck.
See my other answer...
Cannabis tech is where these problems are....
And i have a big network and some amazing work being developed....
It is always satisfying when you can knock out an entire class of problems in one go. Interestingly, this almost invariably involves writing either a parser or an interpreter
Uhm... can i have this app/script whatever???
I have a beautiful problem space for you: the newly legalized cannabis market.
(Btw im doing a cannabis tech consultancy and anyone who is interested in working in this space the ploblem-scope is very interesting)
Please contact me.
Nowadays, wouldn't you just use org.apache.poi for such a task? It can load an Excel file, fill in the input data, and retrieve whatever output data you need, evaluatong and caching formulas as needed.
It's in PHP and really rough by today's standards. We can talk more about it if you wanted to hit me up, my contact info is on my profile.