Home → Problems →

Editing a sensitive/password variable template wipes out the tenant project variable value

• Edit

operations

17 Jul, 2017 06:01 AM

Hi Guys - We've come across what we believe to be quite a serious bug. Over the weekend, one of our Ops guys was adding a new tenant to our system (3.15.5). We have a project variable template for the DatabasePassword variable, which is defined as sensitive/password with no default. The value for this variable is set in the Tenant/Variables/Project Variables screen, however our staff member incorrectly set the password value as a default on the template. This in itself was not a big issue, as all our tenants/environments have this value set for themselves, so the default value would only apply to new tenants. However, upon realising his mistake he removed the default value from the template. At this point, all the previously stored password values for each tenant/environment were wiped out, so each tenant had the orange exclamation mark next to them stating they were missing variable values. He then put the default password back on the template. This cleared up the exclamation marks, but then had the effect of sending out the wrong passwords for the next two deployments before this was realised.

At this point I was called to investigate, and upon finding out what had happened I restored the database to the point prior to these modifications being made, so at this point crisis averted. However, upon further investigation of the sequence of events from the weekend, I have made an alarming discovery. When testing this process with other variable template types (non-sensitive), I discovered no issues, so changing the template default values, or removing them altogether, had no impact on the tenant/environment values set, it only seemed to occur on the password variables. Now I would have thought that specifically-set values for the tenant/environment passwords should not be affected by a change in the default value on the template, whether the default is changed or removed, but I figured that's the way you have designed the password fields to operate. So I thought as a workaround, I'll update the help text on the template variable to advise people not to set or change a default password value on the template variable. I didn't set or change the default variable value at all, just added help text. However, as soon as I saved this change, all the tenant/environment password values were wiped again! I made this change on the production server, so had to restore the database again to just before I made that change.
I recreated this on my local Octopus installation in a new, empty project to confirm this behaviour:

• Added a DatabasePassword project variable template, with no default value set
• Added a new tenant, linked it to this project for a single environment
• Set the password value in the Tenant/Variables/Project Variables screen and tested the deployment - OK
• Added some help text to the variable template, no change to the default value
• Bang! Password value wiped out from the tenant/environment.

This is a serious issue for us, as we have 40 tenants connected to various combinations of 11 environments, so having to re-enter all of these password is not feasible, hence the need to restore the database when this happened. Fortunately we have been able to do this as there had been little activity in Octopus over the weekend when this issue was encountered, but can you please investigate this issue for us? A change to a project variable template should have no effect on explicitly set variable values for a tenant/environment combination. Yes, if a default value is removed then tenant/environment combinations that relied on that default value should get the exclamation mark to notify you that values need to be set for these, but it should not affect ones that have been explicitly set.

Sorry for the long post, happy to provide any furhter information that you may require to get to the bottom of this.

Many thanks, Simon.

1. Support Staff 1 Posted by Daniel Fischer on 17 Jul, 2017 11:23 PM

   

   Hi Simon,

   Thanks for the information here. This was a nasty bug we recently noticed and fixed right away in our latest version 3.15.6. You can see the GitHub issue here:
   https://github.com/OctopusDeploy/Issues/issues/3651

   Could you please update and confirm that this issue is no longer happening?

   Looking forward to hearing from you.

   Best regards,
   Daniel

   • Edit

2. 2 Posted by operations on 18 Jul, 2017 12:27 AM

   

   Hi Daniel - I've installed 3.15.6, but it is still not working correctly. I tested a few different scenarios and was still able to break it. Here's one:

   • Add a Project Variable Template called DatabasePassword with no Help text, and set a Default password
   • Connect a Tenant to the Project for a single Environment
   • No yellow exclamation mark on the Tenant, you can see the Project Variable set for Database Password
   • Edit the Project Variable Template and add some Help text and save
   • You will notice that saving the help text on the template has wiped out the default password value, also causing the exclamation mark to appear against the tenant.
   • I also found that after fixing up the default password on the template, even adding another template also wipes out the default value as soon as it is saved. This means that if we change any of our project variable template values, we will lose any default value set on a sensitive variable.

   I can confirm though, that adding help text to the sensitive variable template no longer wipes out the passwords already set for each tenant/environment, which was what happened yesterday. So that issue is fixed.

   Cheers, Simon.

   • Edit

3. 3 Posted by operations on 18 Jul, 2017 01:03 AM

   

   Attached screen capture showing default password being removed.

   • Project_Variable_Template_Issue.gif 1010 KB
   • Edit

4. Support Staff 4 Posted by Daniel Fischer on 18 Jul, 2017 02:56 AM

   

   Hi Simon,

   Thanks for this. It looks like we fixed this up under Library -> Variable Sets -> Variable Templates but not Project -> Variables -> Project Variable Templates We are looking into this now.

   Here is the GitHub issue: https://github.com/OctopusDeploy/Issues/issues/3671

   Thank you again for all the helpful information here, it definitely helps us resolve these things faster. :)

   Best regards
   Daniel

   • Edit

5. 5 Posted by operations on 18 Jul, 2017 12:08 PM

   

   Awesome, thanks Daniel!

   • Edit

6. Support Staff 6 Posted by Daniel Fischer on 19 Jul, 2017 01:36 AM

   

   Hi Simon,

   No worries, feel free to get in touch at any time. :)

   Best regards,
   Daniel

   • Edit