Queuing Deployment to multiple separate machines.

Daniel Noel's Avatar

Daniel Noel

04 Jan, 2017 11:52 PM

I am auto scaling in aws and I was wondering if there was a way possible to process multiple deployments of a release at the same time instead of queuing a deployment for each machine. (trying to speed things up)

  1. Support Staff 1 Posted by Kenneth Bates on 06 Jan, 2017 01:22 AM

    Kenneth Bates's Avatar

    Hi Daniel,

    Thanks for reaching out! Are you using Project Triggers for auto-deploy? Combining auto-scaling in AWS with auto-deploy Triggers is powerful, though some things to consider:

    Octopus checks by default every 30 seconds for any new events. Auto-deploy allows you to opt-in to run the deployment task once new machines are added (and found in the 30 second intervals). However, if new machines are added while the deployment is executing, Octopus waits for the deployment task to complete before re-queueing the same task for the added machines. It can't run the same task multiple times simultaneously.

    If this looks like your scenario, I'm interested if an idea sounds helpful to you. The 30 second interval is hardcoded into Octopus. Would it be helpful to add a custom setting to increase this interval? That way, if you're auto-scaling quite a bit, it could potentially find more added machines, thus executing the deployment to more machines at a time. However, it would mean Octopus would check less frequently. It may be a great idea to suggest on UserVoice.

    If you aren't using Project Triggers, could you explain a bit more about your process? I'll be happy to reassess if so!

    Kind regards,

    Kenny

  2. 2 Posted by Daniel Noel on 06 Jan, 2017 06:37 PM

    Daniel Noel's Avatar

    It is good to know that I can use project triggers.

    The way I approached this was to setup start up scripts that registers the machine to the project and env and then triggers a deployment to that machine via the api. However I noticed a bottle neck when adding more machines. If another machine gets ready to deploy it would have to wait for the previous machine to finish. It would make sense that if the new machines just need the latest deployment that it would just pull the newest code without having to queue. since this is an on demand kind of action not a deployment to all the machines. [would be a great upgrade in the future if not]

  3. Support Staff 3 Posted by Kenneth Bates on 09 Jan, 2017 06:53 AM

    Kenneth Bates's Avatar

    Hi Daniel,

    Thanks for the extra information about your setup. Octopus will queue any deployments for a single project to the same environment if one is already running. This is on purpose to avoid conflicts with deployment steps/scripts. To get around the behavior, similar to what we do with auto-deploy, is to deploy to as many machines as you need to create and register in a time frame.

    Your point about deploying to individual machines without queuing is valid as it will not have the same issues as deploying to full environments at the same time. We are having an internal discussion about this and its need in a scaling world. I will at least get back to you with a UserVoice suggestion or a ticket for changing this behavior.

    Don't hesitate to get in touch if you have any further questions!

    Kenny

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac