Comment by anirudhrx
2 years ago
Sorry if I'm not using the right terminology here. By layers, I'm talking about the different places from where you're emitting spans - like in your example, the UI, prompt-embedding, vector-store and context-encoding. What I was curious about was two-fold - one whether there are use-cases that make sense to have experimentation of the kind that allows a user to swap out a particular step in that stack - and second about making use of context propagation like baggage enables in OTel.
Got it, appreciate the clarification.
(1) yes, that's what many of our users are doing while swapping might refer to changing model parameters, prompts, embedding stores, embedding parameters, or splitting a "layer" into multiple (parallel) steps. Alternatively when experimenting on a higher "layer", we often see wildly different implementations that are experimented with over time. We added version and release tagging which to build dashboards on the impact of these experiments on quality/latency/cost (currently in alpha, we will release it soon).
(2) I see how metadata propagation could be much more elegant via OTel. Currently the metadata fields on traces/span and filtering based on them kind of does the job but I see how it could be much easier to ingest and use. Thank you!