Comment by adam_rida
4 days ago
thanks to everyone for taking the time to try Echo and share feedback, this is precisely why i wanted to launch early.
i am going to try to address a couple of topics that came up often:
- i'll keep publishing stronger evals, including more difficult coding and agentic benchmarks, to map out more precisely the differences with sota
- the public eval dashboard will keep expanding and be updated (very open to more benchmark suggestions as well!)
- some people found issues in the eval dashboard ui and the sign up flow, should be now all fixed in prod
some important precisions as well:
- NO credit card is required to try Echo
- each acount includes 10$ of free credits to try on both the API and the chat
on the approach itself: the idea i'm exploring is more broader than model routing, i'm looking at how to allocate inference efficiently across open-weight models, deciding not only which models to use, but also how much computation a request deserves and how intermediate work should be combined.
ensembling by itself is not new. since random forests and probably even before in statistics/classic ml we knew that bringing multiple models together can outperform individual ones. the interesting problem for Echo is how to model and leverage this without paying the full ensemble cost at each request.
while there are conceptual similarities with systems like Fusion or Fugu, the architecture and optimization objective are different.
thanks again for all the thoughtful feedback.
Small feedback: the "create password" requires a symbol too, which Google's password manager by default does not use. I'm fairly sure a double-digit-level alphanumeric jumble is sufficient to be a password (or at least, Google thinks so). Great idea nonetheless!
> I'm fairly sure a double-digit-level alphanumeric jumble is sufficient to be a password
This isn't required either. Of course there's an xlcd for that: https://xkcd.com/936/
Besides,
--- start quote ---
Using complexity requirements (that is, where staff can only use passwords that are suitably complex) is a poor defence against guessing attacks. It places an extra burden on users, many of whom will use predictable patterns (such as replacing the letter ‘o’ with a zero) to meet the required 'complexity' criteria.
https://www.ncsc.gov.uk/collection/passwords/updating-your-a...
--- end quote ---
> This isn't required either. Of course there's an xlcd for that: https://xkcd.com/936/
Oh, I'm well aware (with the concept and comic both). My passwords that I set for myself almost always are like that. It's just that I wasn't sure if OP was aware of it, due to what their site was asking.
This. The siren song of "oh but increasing the number of characters makes brute forcing harder exponentially" is defeated in practice by the fact that everybody's password is following a few predictable patterns. You know what else increases the complexity exponentially? Longer passwords
1 reply →
Seconded
https://blog.codinghorror.com/password-rules-are-bullshit/
thanks! adjusting it now
It doesn't let me sign up. Throws `too many authentication attempts` error. There was only one attempt.
same here
works now
1 reply →
Why the dark pattern?
You had my interest, now not anymore.
Remove the dark pattern
You keep saying open weights - but you aren't sharing any information on which models you are using. What benefit does using open weights models provide to the end user if there is zero transparency?
I guess it’s their IP