Dynamically creating variables to be used in JSON Configuration Variables feature

jonas.cannehag's Avatar

jonas.cannehag

10 Jan, 2018 10:42 AM

Hi, we are using Octopus for many of our deployments. We are going to centralize our configuration in a custom build service.
What I would like to do is to have a PS-script getting the configuration as a JSON block and then in some way have those values inserted using the "JSON Configuration Variables" feature.

What is causing issues is the PS-script. I can get the values without a problem, but the JSON feature seems to only respect Octopus "predefined" variables, which I can't modify... The only thing I can do in my PS-script is to use the Set-OctopusVariable cmdlet, which in turn just creates the output variable on that step.
Is there any way to add variables in a script that is respected and used in the JSON config feature ?

  1. Support Staff 1 Posted by Kenneth Bates on 11 Jan, 2018 01:29 AM

    Kenneth Bates's Avatar

    Hi Jonas,

    Thanks for getting in touch! Using the syntax #{Variable} will only match against project variables (or in a library). To substitute output variables, you would need to target the step that creates the variable using the syntax #{Octopus.Action[StepName].Output.VariableName}. (This is assuming the step that creates this variable runs prior to the step that substitutes the value).

    JSON substitution targets JSON objects by using a special syntax, like foo:bar, which is outlined in the following doc page.
    https://octopus.com/docs/deploying-applications/deploying-asp.net-c...

    I was able to get this working by creating an output variable in a script step, targeting the file in the json substitution feature, and create a project variable with name foo:bar, where the value of this project variable uses the output variable syntax. I've included a few screenshots showing my set up.

    I hope this helps! Let me know how you go, and if you have any further questions, don't hesitate to reach out. :)

    Best regards,

    Kenny

  2. 2 Posted by jonas.cannehag on 11 Jan, 2018 06:00 AM

    jonas.cannehag's Avatar

    Hi and thank you for responding.
    I think you are missing the word dynamically in the title. What you are suggesting is to hardcode every configuration value which is what we do today.

    What we are trying to accomplish is to have the config elsewhere but make use of the JSON config feature.

    As an example, I can retrieve a JSON object from our external config service looking like this:
    {
       "foo": "bar",
       "also:working:with:depth": "baz"
    }

    I want make use of this as input to the JSON Config feature somehow. Do you have any other idea if that is possible without hardcoding the names, because that should be up to the config service, not Octopus

  3. Support Staff 3 Posted by Kenneth Bates on 15 Jan, 2018 04:43 AM

    Kenneth Bates's Avatar

    Hi Jonas,

    Thanks for following up! I used a basic example of how to store an output variable value which you can then use to substitute into your JSON. You could expand upon it to have a script that runs some custom code, and store the resulting value in an output variable in the same way. As a simple example of this, say you have a script step that runs:

    $variable = "#{Octopus.Environment.Name}"
    
    Set-OctopusVariable -name "NewVar" -value "$variable"
    

    Then when calling it in the subsequent step with:

    write-host "#{Octopus.Action[output script].Output.NewVar}"
    

    It writes out the relevant environment I'm deploying to.

    We have some additional information on what can be done with output variables in the following blog post.
    https://octopus.com/blog/fun-with-output-variables

    Does this help? Don't hesitate to reach out if you have any further questions. :)

    Best regards,

    Kenny

  4. 4 Posted by jonas.cannehag on 15 Jan, 2018 06:13 AM

    jonas.cannehag's Avatar

    Hi, thank you again! I think I need to explain a bit more :)
    The problem is that I don't now anything about the variables, not the name, not the value and not how many they are. I will just get a list of keyvalue pairs from our configuration service and would like to append those as "normal" variables so they will be used by the feature.
    A script to generate output variables is not a problem. The problem is to get them to be used by the JSON configuration feature...

  5. 5 Posted by jonas.cannehag on 30 Jan, 2018 12:20 PM

    jonas.cannehag's Avatar

    Hi again, any ideas or suggestions on my last reply ?

  6. Support Staff 6 Posted by Kenneth Bates on 09 Feb, 2018 01:31 AM

    Kenneth Bates's Avatar

    Hi Jonas,

    I'm terribly sorry about the delay in getting back to you. I must have hit the wrong button when I tried to respond to you last week - it looks like that was completely my fault. Thank you for following up on it to reactivate this thread. I meant to ask a few followup questions which I'll be sure to ask here. :)

    Generally we suggest including everything that's needed for your app to run to be included in the package, so dynamically adding to files like this on the fly from an external service may be tricky. If you have a script to perform some logic and retrieve values, could you store the needed information in variables in your script and then use some additional logic to define your output variables that way?

    What have you tried up to this point? I fear I may still be misunderstanding your scenario a bit, unfortunately, so I may reach out to my team for further help if needed.

    Kind regards,

    Kenny

  7. 7 Posted by jonas.cannehag on 09 Feb, 2018 06:24 AM

    jonas.cannehag's Avatar

    No worries, just eager trying to get this to work. Let me try to explain this a bit more.
    We have many projects in Octopus and would like to avoid having all of our configuration there for different reasons. What we have in mind is centralizing config into an external service where we can have more functionality provided to our services.

    The one feature that is the problem here is the "JSON Configuration Variables".
    We are using json configuration files in our services and this features works really well for replacing a json structure. For example adding a variable in Octopus with the name "AppSettings:AnotherService:Url" will replace that structure in our appsettings.json file. Works really well!

    But, I would like to move that variable into something else. A configuration service that will expose a list of key values like this
    [
       "AppSettings:AnotherService:Url": "http://.....",
       "AnotherSetting:SomeStructure:SomeProperty": "..."
    ]

    Now, I would like to have this array been used in the JSON Configuration Variables feature in some way, like "append" to the Octopus variables list. Do note, that nothing can be hard coded here. Octopus should have NO knowledge of what the array contains.

    I could use a script to get the values, that's not the problem. How do I get these values into the feature so it will modify the appsettings.json file? From what I understand, that feature will not be able to replace variables when they are output variables from another step right ?

  8. Support Staff 8 Posted by Kenneth Bates on 14 Feb, 2018 02:23 AM

    Kenneth Bates's Avatar

    Hi Jonas,

    Thanks for following up. You're right, output variables won't be able to be substituted into JSON files with the JSON configuration variables feature, and I've done a test to confirm that. The variable has to exist as a project or library variable when the release is created (so it's included in the snapshot). You can define the values of these pre-created variables to have a dynamic value calling evaluated output variables from a previously run step in the deployment (i.e. a variable named foo:bar with a value of #{Octopus.Action[stepName].Output.VariableName}, but unfortunately that's the extent of this feature. Values that are needed to be modified in the JSON file would need to exist in Octopus.

    Perhaps the substitute variables in files feature would get you further to your goal. This feature is a bit more flexible, and you can store a lot of information within a single variable. Theoretically, you could have some logic in your script to retrieve all of these needed values, store the result in a single output variable, and call that variable in your specified JSON file. If your JSON file only has this variable during deployment, and you dynamically retrieve this information and substitute it in, I would think that could get you closer to what you're after, but obviously it means you wouldn't be using the JSON configuration variables feature.

    You can read more info on this feature in the following documentation:
    https://octopus.com/docs/deployment-process/configuration-files/sub...

    If you're interested in trying this option out, you can create a single project variable (something like JSONConfigValues) with a value calling the evaluated output variable. In the JSON file, specify #{JSONConfigValues} and that will be replaced on deployment.

    Sorry it's not better news! Please let me know how you go and if you have any further questions. :)

    Best regards,

    Kenny

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac