Comment by SdtEE
2 months ago
This starts from my frustration when opening large CSV files, but later evolved to a log/data analyzer that loads arbitrary format in constant time (O(1)).
https://github.com/Verticalysis/Hitomi
The secret: I engineered an incremental combinatorial parser capable of processing customized format from a steam. Any inputs, including file or the stdout from a command, are first chunked and then fed to the pipeline. The UI is ready when the first small chunk is processed.
Other highlights: 2-mode filter, one with a convenient UI and the other is based on an extensible DSL for complex cases;
Timeline mode scrollbar, a secret weapon for log or time series analysis;
Column widths fit to content automatically;
Native code, no web bloat;
Cross-platform (currently Windows and Linux, MacOS WIP).
If you are tired of all the quirks Excel have when working with CSV files, you'll gonna love it!
Does it update the parser incrementally?
I'm not sure what do you mean by "update". If you mean the state of the parser - of course, and I did some trick to reduce the supposedly infinite states for arbitrary format to a fixed set of 4. If you mean by the parser program it self -No. The parser was developed with Dart, a statically compiled language, with no ability to modify the program itself.