Comment by XTXinverseXTY
16 hours ago
OP's was leaky slop from day one [0][1], as is his article [2]
It is arrogant and entitled for the author to take credit for the concept of RL over sequence embeddings, and none of the work that went into pretraining, not to mention the egregious target leakage [1]
[0]: Author fails to grasp the concept of virtual environments https://www.reddit.com/r/LocalLLaMA/comments/1kl0uvv/comment...
[1]: his `train.py` has `outcome` as a model input (conversation_metrics built from _parse_conversation which includes outcome): https://huggingface.co/DeepMostInnovations/sales-conversion-... https://huggingface.co/DeepMostInnovations/sales-conversion-...
[2]: 100% of this post is AI-generated https://www.pangram.com/history/97e0be84-391d-46b8-9c16-2d8f...
For anyone else verifying, the target leakage appears to be as follows:
1)`outcome` is part of `metrics` at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
2) `metrics` goes into `ConversationState` at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
3) `metrics` (including `outcome`) makes its way into `ConversationState.state_vector` at https://huggingface.co/DeepMostInnovations/sales-conversion-..., and is returned from environment `step()` and `reset()` functions at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
4) model ingests `state_vector` as input at https://huggingface.co/DeepMostInnovations/sales-conversion-...