Comment by piltboy
19 hours ago
Generally speaking I would classify TIFF and its variants as imaging formats (or for very simple numerical datasets), and NetCDF as more suited for raw data, in particular multi-dimensional data with time series, etc.
For forecast and climatological data I find NetCDF is vastly superior, but also much more complicated to work with due to the capabilities and how open the format is. Just have a look at the complexity of the CF Conventions to see what I mean: https://cfconventions.org/cf-conventions/release/v1.12.0/cf-...
For visualizing orthophotos and the like, I would choose GeoTIFF any day of the week, as they're easy to visualize across platforms using existing libraries. Using COGs you also get the functionality of a spatial index within each GeoTIFF file, meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
Yeah the complexity of NetCDF was the ick factor that made me use GeoTIFF. It's just not the level of complexity I wanted to deal with in-program. I didn't want to wade through the details to understand the format, and I didn't want to then lock users to some subset
Seemed easier to let users preprocess their NetCDFs into GeoTIFFs manually. I have a bunch of hacky scripts to massage NetCDFs from different sources in to compatible GeoTIFFs
> meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
Interesting. My performance bottleneck right now is the user selects a small regions and then the program has to read in GBs of global precipitation maps (from IMERG) and cut out tiny squares. In the extreme cases it can mean ~2 minutes of waiting for a result. This means the user can't casually select and try out different regions with quick feedback. If you have a beefy machine you can keep it all in RAM sometimes and it works better.. but it's not ideal (my 16GB machine can only handle simpler scenarios)
I'll take a closer look at in the future. At the moment I just use Java's default TIFF reader and ImageBuffer class. Maybe it'd be easy to convert to COG format and adapt in a COG reader