Comment by goku12
16 hours ago
It's the use case. Here is one concrete example. I worked as a project engineer during the development of a launch vehicle. The telemetry data frames from every test and every flight were processed into numerous CSV or TSV files that were labeled with the parameter name. Those files could be very large depending up on their sampling rates, especially for tests that lasted hours on end. You would conduct exploratory manual analysis on that data which involves:
* Quickly cycle visually through time series graphs (often several hundred parameters). You'd have seen most of those parameters before and would quickly catch any anomalies. You can clear so much data rapidly like this.
* Quickly analyze a graph at various zoom and pan settings. May be save some as images for inclusion in documents. Like above, the zoom and pan operations often follow each other in a matter of seconds.
* Zoom into fine details, down to single bit levels or single sample intervals. There's surprising amount information you can glean even at these levels. I have run into freak, but useful single events at these levels. And since they're freak events, it's hard to predict in advance where they'd show up. So operation speed becomes a key factor again.
* Plot multiple parameters (sometimes with different units) together to assess their correlation or unusual events. We used to even have team analysis sessions where such visualizations were prepared on demand.
* Do statistical or spectral analysis (like periodograms, log or semi-log graphs, PDFs, etc)
* Add markers or notes within the graph (usually to describe events). Change the axes or plot labels. Change grid value formatting (eg: Do you want time in seconds or HMS?).
All the operations above are possible with Julia, Matlab, R or Python. And we did use almost all of them (depending on personal preference). But none of them suit the workflow described above for one simple reason - speed. You don't have enough time to select each parameter by text or GUI. There must be a way to either quickly launch a visualization or cycle through the parameters as the investigator closes each graph. You also don't have time to set zoom, pan and labels by text. It must be done using mouse (zoom & pan) and directly on the graph (labels and markers) in a WYSIWYG manner. And you don't want to run an FFT or a filter function, save the new series and then plot it - you want it done with a single menu selection. The difference is like using a C++ compiler vs Python in JupyterLab. The application we used was very similar to Labplot.
Now, Excel might seem like a better choice. In fact, LabPlot and our application all has a spreadsheet-like interface with the ability to directly import CSV, TSV, etc. But Excel just doesn't cross the finish line for our requirement. For example, to plot a time series in excel, you have to select the values (column or cells), designate the axes, optionally define the axes and graph labels, start a plot, expand it to required levels and format the print. At that rate, you wouldn't finish the analysis in a month. Those applications would do all that on their own (the labels and other metadata were embedded in the data files by means of formatted comments). But an even bigger problem was the size of the data. Some of those files on import would slow down Excel to speed of molasses. The application had disk and memory level buffering to significantly improve the responsiveness to almost instant interactivity.
I hope this gives you an idea where the tools that you mentioned are not good enough replacements for LabPlot and similar tools.
Sounds like plotjuggler (https://github.com/facontidavide/PlotJuggler) could be worth checking into as well for you.
I'm also space and launch-vehicle adjacent. Using vnlog for data storage (like what you described, but with better tooling support) and feedgnuplot/gnuplotlib for visualization. Works great. The learning curve is really easy, you can get going and start analyzing stuff FAST. Making complex plots is fiddly, but it usually is with any tool.
Thank you for this fantastic elaboration. I am in a very similar boat (unmanned aerospace) and have very similar needs. I’ve been chewing on making my own application to do this but LabPlot looks like it has potential to be exactly what I’ve been dreaming about for a few years.