Comment by jordanbeiber
3 years ago
Wholeheartedly agree - convention before configuration wins with tooling even at modest scale.
Larger team(s) using free form yaml configuration and templating quickly becomes messy.
Setting a boundary at “configuration” and exposing configuration options to dev teams through settings works really well.
It requires you to have a tooling team with a few developers, but it scales!
Convention over configuration is really the future, but not everything follows that philosophy. For example, OCaml codebases are basically a free for all and you can organize things however you want. So with that in mind, you need something to organize and build.
Convention seems to be nice in the beginning. But it's very hard to maintain.
Much better is default configuration. I.e. all "conventions" are explicitly generated into a default-configuration file which you can then change/overwrite/update in whatever way you want.
I think you can have both (for example, go and rust starter projects)
One problem with convention is that it isn't discoverable. At least config files guide you to where to look for stuff.
But even so I think you're probably right that convention is better because it forces everyone to use the same structure for stuff.
Convention before configuration doesn't solve the core problem.
Why do you need configuration? Because you build your application by gluing together a multitude of different applications that need to configured properly to be able to talk to each other.
Once you understand this, the solution is obvious: don't build your application this way.
Build it in one language. If you want to "reuse" existing solutions, reuse them as libraries, not as separate programs.
Say sql database:
Teams have agreed to use postgres for sql, this is by agreed convention.
A tooling team implement hashicorp vault and build a pg module/deployment that is packaged and accessible through whatever tooling you build.
This makes a datasource of type pg sql available to all teams in a couple of minutes.
These modules/deployments are governed by a bunch of conventions, but those are of no special concern to consuming dev teams.
Use “something” to keep track of service upstream/downstreams: consul, a database, or something else.
When a service is connected to a database, say using consul upstreams you have the job gen api inject the appropriate env vars according to an agreed upon convention.
Now dev teams have an option to deploy incredibly secure pg databases, and by simply setting a service as downstream it will automatically receive rotating connection strings.
There’s no configuration to be managed by dev teams except keeping a service graph up-to-date.
Lot of conventions above, and a tooling dev team with a lot of infra and tooling expertise.
I believe that for scaling dev teams you should look at tooling like a product domain and treat it as such.