Comment by jonhohle

1 day ago

Are most feature flag systems really just simple Boolean checks like this with no feedback loop? In 2009 Amazon’s equivalent supported multi-variate flags (run test with more than 2 options), a feedback loop for performance/attribution, and weighting. I don’t see any of that on the unleash home page, but also haven’t seen it in other feature config systems either.

7 comments

jonhohle

Reply
hobofan  1 day ago

What you are describing is usually part of analytics platforms (Mixpanel, etc.). A feature flag toggle only plays a small part in this (but it may also be used for a lot of other scenarios of how you may want to control feature flags).

Audience is decided by analytics tool -> Feature flag server maps audience to feature flag value -> Analytics tool tracks its events and includes feature flag value as attribute

  • jonhohle  1 day ago

    So if you want to slowly roll out a feature flag, you need to get a different app involved that can do everything the feature flag tool can do and a whole lot more? What’s the point of the feature flag tool?

    Analytics is a much broader set of tooling. It can (and should) report similar things.

    • hobofan  1 day ago

      The point of the feature flag tool is to control feature flags. They are not just for A/B testing and slow rollouts, and most products are not on the scale of Amazon, where you need to have a feedback loop for the rollout. Of course there is always going to be some overlap between tools, where many analytics platform allow you to do the rudimentary version of this.

      The main point of a FF system is that you have a runtime-controllable mapping from user attributes to feature flags, where a set of logic rules can dictate who is exposed to what feature. With most analytics-tool based rollouts, you would be able to only directly set an attribute and you then hard-wire that attribute to the feature toggle in app code, which gives you a lot less flexibility and always requires a engineer in the hot path when it comes to doing rollouts.

horticulturist  1 day ago

usually some “data science” team takes the results and decides what to do. really heavy process for something that is usually quite simple.

  • wmichelin  1 day ago

    We have automatically generated readouts that run hourly and daily, show metrics between control and variant across a ton of core business metrics, and we can even opt into measurement of some super esoteric stuff.

    Granted we have a whole team supporting just that experimentation platform, but it's pretty cool. They even expose an MCP server for us to have agents do experiment analysis for us using.