Comment by idosh
2 years ago
Congrats on the launch! Sounds like an exciting project. Do you plan to store also the raw data (input + output)? It can be relevant for fine-tuning, optimizing costs, etc. Since you already store metadata, I think it makes sense to have a one-stop shop.
Agree – Langfuse stores all prompts/completions, model configuration and metadata. Currently the GET API can be used to use the data for finetuning and we build a wrapper to access a filtered sample via the Python SDK.