Support users upgrading configurations deprecated in 0.5.0

[lornasong]

Description

There were a number of configuration deprecations introduced in 0.5.0 (details) to make the configuration more consistent.

These deprecations are:

• Deprecate task condition block field "source_includes_var" for "use_as_module_input" #568
• Deprecate task configuration "source_input" block for "module_input" #567
• Deprecate task configuration field from "source" for "module" #566
• Deprecate task configuration "services" field #669
• Deprecate service block configuration #670

This is quite a few deprecations and some of the upgrades to replace the deprecations are not quick changes. We want to provide some way for users to have an easier experience to upgrade their configuration.

Proposal:

Tentative proposal - subject to change.

Provide a CLI for users to upgrade their configuration. For example:
consul-terraform-sync config upgrade -config-file=config.hcl

Potential CLI flags/options for how to do the upgrade:

• Go ahead and make the suggested upgrade to the users configuration
• Don't modify the users configuration but output the suggested upgrade to the terminal
• Save the suggested upgrade to a secondary file

[jyipks]

having the config upgrade sounds excellent, would having CTS monitor the config-file / config-dir be sufficient.. and if any changes to those files it would reload? currently whenever i add a new service to the config file, i have to restart the daemon.

There is also no way to remove all services from what i've seen so far, being able to "destroy" everything would be a nice feature for clean up if i ever want to just take it all down.

i am currently using 'cts_user_defined_meta' to provide information for our network loadbalancer, providing IP, PORT, NAME. These are all unique per service so its extremely useful for the network operator to configure this and have terraform configure the rest accordingly.

i personally prefer having multiple service blocks and defining the names in a single task block. Because i need 'cts_user_defined_meta' and it is unique to each service i will need to have an absurd amount of task blocks after the deprecation.

[lornasong]

Thanks for the great feedback and information about how you use CTS @jyipks!

To answer your question about reloading CTS with the upgraded config - the current proposal is to only generate new configuration. Users would still unfortunately have to restart CTS to have it use the new configuration. CTS unfortunately doesn’t have a config reload or update task feature yet.

Regarding a way to destroy everything - that’s good to know. We have a Delete Task CLI which only deletes the task from CTS in memory but doesn’t destroy any generated files or resources. If you’re open to it, we would welcome a new GitHub issue capturing your use-case and the details of what you’d want to destroy. Note: similarly a new issue on updating tasks during runtime, but only if you're up for it!

Thanks for all your information about how you use cts_user_defined_meta. That's really helpful to know that you prefer the multiple service blocks. I’ll create an internal ticket with this feedback. We’ll revisit the decision to remove service block before supporting an upgrade tool.

Let me know if you have any questions, thoughts!