keys are inconsistent with different backends

[rogpeppe]

When using the env backend, then the only keys considered will be those with config: field tags. However, if you're using a file backend, then the entire struct is unmarshaled from the source, and all fields will be considered, even those without a config tag.

So it's possible there are some fields which are configurable only with the file backend and not with the env backend.

I think that probably that any keys that don't specify the config: tag should be ignored so backends are more consistent.

[rogpeppe]

To go into a bit more detail on this:

The confita README says:

Confita scans a struct for config tags and calls all the backends one after another until the key is found. The value is then converted into the type of the field.

and then, later on:

As a special case, if the field tag is "-", the field is always omitted. This is useful if you want to populate this field on your own.

This description is true for all the backends except the file backend, which instead unmarshals the file data directly into the struct, meaning that the config tags are entirely ignored.

This has a few implications:

• some values are only configurable in certain backends. For example, in the kafka-proxy service, it's possible to configure the proxies with the file backend but not using any of the other backends.
• some types are not consistently treated. For example, a time.Duration field will accept 4h30m in all backends except when using JSON or TOML in the file backend, where you'd need to use 16200000000000.
• fields that are explicitly marked with config:"-" are not ignored when using the file backend. For example, in the kafka-go component, the Consumer field is marked to be ignored, but will actually be unmarshaled when using the file backend.

This issue won't be trivial to fix in a backward compatible fashion, as making field names consistent would break some existing behaviour.

Here is some code that demonstrates some of the issues: https://play.golang.org/p/UJcNsbP6NOc

[yaziine]

Good catch @rogpeppe.

As you said it will be complicated to fix that without breaking the actual API.

Do you have any idea in mind?

[rmcodor]

I would like to work on this issue, any guidance or directions?

[tealeg]

@rmcodor we're actually about to branch the current codebase into a V0 branch and work on a major restructuring should make things more consistent. I'd hold off before diving into anything too big. Are you looking to work on it because it's an issue that affects you, or are you just looking for things to do? If it's the latter we can chat about that too.