Using Chef to automate Octopus Deployments

If you are using Octopus Deploy to deploy your .Net code to your Windows servers and also using AWS Auto Scaling Groups, you may have come across some of the limitations of Octopus. Primarily, Octopus has no built-in process to deploy the current project release to a newly registered tentacle immediately upon the tentacle registration. Tentacles can either be registered as 'polling' or 'listening' tentacles.  Polling tentacles will periodically check with the Octopus server to determine if a deployment is required, while a listening tentacle will wait until a deployment is pushed from the Octopus server.  Neither of these approaches will deploy the release immediately.  This post will provide a process of using a Chef recipe to configure the Octopus tentacle in listening mode and then initiate a deployment of the latest project release from the Octopus Server using an API call.

I must give credit to CodeKing and his article of how to use Octopus Deploy with AWS.  His script provides the steps necessary to query the Octopus server for the latest project release and to initiate the deployment. Here is his post:
http://www.codeproject.com/Articles/719801/AWS-Deployment-With-Octopus-Deploy

The process I will demonstrate will rely on Chef to register the Octopus Tentacle as well as initiating the deployment of the latest project release to the server.  I'm not going to go into details about using Chef, if you need help using Chef please refer to their documentation at: https://docs.chef.io/

The recipe assumes the following:

• Amazon's AWS CLI is installed on the server (this can be done using another chef recipe)
• The server has access to S3 to download files to install Octopus Tentacle
• The Octopus project is configured to increment version numbers
• An Octopus account with an API key that has proper permissions to deploy the release.

The recipe will perform the following tasks:

1. Download the Octopus Tentacle installer.
2. Install Octopus Tentacle
3. Register the Octopus Tentacle to the Octopus Server
4. Deploy the latest project release from Octopus Server

1. Download the Octopus Tentacle installation from S3

It's not necessary to put the installer file on S3, but I prefer this approach as I know the file will always be available, vs relying on a web link that could potentially change.

Modify this script to use your S3 bucket
powershell_script 'Download Octopus Deploy' do code <<-EOH set-location c:/temp read-s3object -bucketname YourS3Bucket -key Octopus.Tentacle.3.1.2-x64.msi -file OctopusTentacle.msi EOH not_if do ::File.exists?('C:\temp\OctopusTentacle.msi') end end



2. Install Octopus Tentacle
With the installer downloaded, we must now install Octopus on the server.

windows_package "Install Octopus Deploy" do source "C:/temp/OctopusTentacle.msi" options "/qn" installer_type :msi action :install not_if do ::File.exists?('C:/Program Files/Octopus Deploy/Tentacle/Tentacle.exe') end end


3. Register the Octopus Tentacle to the Octopus Server
The next step is to configure and register the Octopus tentacle to the Octopus server. This will register the tentacle using the server hostname, which is later used for the deployment process.

Modify this script to use your Octopus server, API key, and role
batch 'configure octopus' do code <<-EOH cd "c:/Program Files/Octopus Deploy/Tentacle" Tentacle.exe create-instance --instance "Tentacle" --config "C:/Octopus/Tentacle.config" --console Tentacle.exe new-certificate --instance "Tentacle" --if-blank --console Tentacle.exe configure --instance "Tentacle" --reset-trust --console Tentacle.exe configure --instance "Tentacle" --home "C:/Octopus" --app "C:/Octopus/Applications" --port "10933" --noListen "True" --console Tentacle.exe register-with --instance "Tentacle" --server "http://octopus.domain.local" --name #{node.hostname} --apiKey "abcdefg12345678" --comms-style "TentacleActive" --server-comms-port "10943" --force --environment #{node.chef_environment} --role "webserver" --console Tentacle.exe proxy --instance "Tentacle" --proxyEnable "False" --console Tentacle.exe service --instance "Tentacle" --install --start --console EOH not_if do ::File.exists?('C:/Octopus/Tentacle.config') end end


4. Deploy the latest release
The final step is to query the Octopus server for the latest release of the specified project and then make an API call to the Octopus server to initiate a deployment to the server.

Modify this script to use your Octopus server, Project, and API key,
powershell_script 'Octopus Deploy Latest Release' do code <<-EOH #Sets an Alias for Octo.exe Set-Alias Octo "c:/chef/cookbooks/octopus/files/octopus/Octo.exe" -scope Script #Get the latest release $release = (Octo list-latestdeployments --project="OctopusProject" --server="http://octopus.domain.local" --apikey="abcdefg12345678" --environment="#{node.chef_environment}" | ?{$_.contains(" Version:") } | select-object -first 1) #Trim the $release variable to just the relase number $release = $release -replace "\\s+Version:\\s+", "" #Deploy the latest release for the project to this server $deploy = (Octo deploy-release --project="OctopusProject" --server="http://octopus.domain.local" --apiKey="abcdefg12345678" --releaseNumber="$release" --specificmachines="#{node.hostname}" --deployto="#{node.chef_environment}" --waitfordeployment --deploymenttimeout="01:00:00") #If the deploy fails, return the error if ($deploy -match 'Exit code:'){ Write-Host "Error occurred: " $deploy } EOH end


Running Chef - putting it all together
The above steps can be placed into a single Chef recipe or kept into separate recipes and run individually.  I prefer to put these in a single recipe and call it via the Chef run-list.

I currently use Chef-Solo and therefore make a call like this, specifiying the runlist as well as the environment.
chef-solo -c c:/chef/solo.rb -j c:/chef/runlist.json -E development -L c:/chef/log.log -l info

AWS - using SQS to cleanup Active Directory of terminated instances

If your Amazon EC2 instances are part of an Auto Scaling Group and are required to be joined to a Windows Domain, then maintaining a clean Active Directory environment may be an after thought. EC2 instances can be terminated for a variety of reasons, and since they may terminate abruptly their Active Directory objects may not be removed from Active Directory.  The following steps will help you create a process utilizing Amazon's Simple Queue Service (SQS) to remove terminated instances from Active Directory and to help keep a cleaner Active Directory structure.

The following assumes:

• Your servers are launched using an Auto Scaling Group and are auto joined to an Active Directory domain using the AWS Instance Id as its hostname.  Please see this post for details on how to accomplish this -  Auto Join EC2 instances to domain
• An EC2 windows instance that has
• An IAM role assigned to the instance
• The AWS CLI installed on the instance
• Access to the Active Directory domain

1. Create the SQS queue

Within AWS, create a new SQS queue.  Be sure to set the message retention period to a value greater than how often you plan to run the scheduled powershell script. We will set the permissions in a later step, after we've created the SNS topic.

2. Create the SNS topic

Create a new SNS topic in AWS and add a subscription to the SNS topic selecting 'Amazon SQS' as the endpoint, ie: arn:aws:sqs:us-east-1:123456789012:SQS-InstanceTerminations

3. Configure the SQS queue permissions

Return to the SQS created in the prior step and select the Permissions tab.  Add/Modify the permissions to allow SQS:SendMessage from the SNS topic you just created.  Modify the below policy to use your SNS ARN and the SQS ARN resource.

{ "Version": "2012-10-17", "Id": "SQSInstanceTerminationsPolicy", "Statement": [ { "Sid": "SQS-Access", "Effect": "Allow", "Principal": "*", "Action": "SQS:SendMessage", "Resource": "arn:aws:sqs:us-east-1:123456789012:SQS-InstanceTerminations", "Condition": { "ArnEquals": { "aws:SourceArn": "arn:aws:sns:us-east-1:123456789012:SNS-InstanceTerminations" } } } ] }

4. Configure the notification for the Auto Scaling Group

Select your Auto Scaling Group and choose the 'Notifications' tab and then 'Create notification'.
For the notification choose the option 'terminate' and select the SNS topic created earlier.

5. Configure the IAM role

The EC2 instance that will be running our Powershell cleanup script  requires permissions to access the SQS queue.  To allow this, configure a security policy for the IAM role that is attached to the instance.  Modify the policy below for the Resource ARN to match your SQS ARN.

"SQS-Access": { "Version": "2012-10-17", "Statement": [ { "Action": [ sqs:GetQueueAttributes, sqs:GetQueueUrl, sqs:ReceiveMessage, sqs:DeleteMessage ], "Resource": "arn:aws:sqs:us-east-1:123456789012:SQS-InstanceTerminations", "Effect": "Allow" } ] }

6. Create the Powershell script to retrieve the SQS messages

Powershell is used to obtain the SQS messages of the terminated instances and then removes the terminated servers from Active Directory.  Save the script on the server that will run the scheduled task.

Here is the script for the complete process. Modify this script to use your SQS queue name.

#Get the SQS queue URL $queueurl = Get-SQSQueueUrl -queuename SQSQueueName -region us-east-1 #Get the number of SQS messages in the queue $messages = Get-SQSQueueAttribute -QueueUrl $queueurl -AttributeName ApproximateNumberOfMessages -Region us-east-1 $messageCount = $messages.ApproximateNumberOfMessages #Loop through each message to remove the terminated server from Active Directory While ($messageCount -gt 0) { $messageCount-=1 $message = Receive-SQSMessage -QueueUrl $queueurl -Region us-east-1 $messagebody = $message.Body.Split(",") $id = $messagebody[19].TrimStart('\"EC2InstanceId\":\"') $id = $id.TrimEnd('\"') Get-ADComputer -Identity $id | Remove-ADObject -Recursive -Confirm:$False Remove-SQSMessage -QueueUrl $queueurl -ReceiptHandle $message.ReceiptHandle -region us-east-1 -Force }

7. Create a scheduled task to run the Powershell script

To schedule the script,  configure a scheduled task on the Windows EC2 instance to run "Powershell" with an optional argument. 

The program path for PowerShell is: 'C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe'

The optional arguments is the path to your script:  'C:\Scripts\ActiveDirectory-CleanUp.ps1'

NOTE: You must run the scheduled task using a Windows User account that has the appropriate user permissions to remove objects from Active Directory.