Comment by thebuilderjr 2 years ago [flagged] 2 comments thebuilderjr Reply Hugsun 2 years ago How do you infer your Parquet schemas? threeseed 2 years ago You infer the types of the source data.For example you can go through say 1% of your data and for each column see if you can coerce all of the values to a float, int, date, string etc. And then from there you can set the Parquet schema with proper types.
Hugsun 2 years ago How do you infer your Parquet schemas? threeseed 2 years ago You infer the types of the source data.For example you can go through say 1% of your data and for each column see if you can coerce all of the values to a float, int, date, string etc. And then from there you can set the Parquet schema with proper types.
threeseed 2 years ago You infer the types of the source data.For example you can go through say 1% of your data and for each column see if you can coerce all of the values to a float, int, date, string etc. And then from there you can set the Parquet schema with proper types.
How do you infer your Parquet schemas?
You infer the types of the source data.
For example you can go through say 1% of your data and for each column see if you can coerce all of the values to a float, int, date, string etc. And then from there you can set the Parquet schema with proper types.