← Back to context

Comment by gavinray

5 hours ago

Because those tests are the only ones that prove the connector is working as intended

Our connectors receive a generic query IR from a Datafusion-based engine

Each connector is then responsible for faithfully translating the query IR into the semantically equivalent query for the underlying data source

For a SQL DB this would be a SQL query, for MongoDB a JSON predicate query, for csv/json files it would be an in-memory predicate engine, etc

Running the generated query against the target data source and asserting that this doesn't throw an error and that the records returned are both correct and in the right order is the true test