AWS Lambda function to remove terminated instances from Octopus server

Octopus Deploy provides machine policies which allows you to configure your Octopus server to remove machines automatically, but it only removes the machine if it is considered unhealthy.  This is normally ok, but what if you want the cleanup to be near real-time? Especially if you launch alot of instances using AutoScaling Groups.  Or, what if you need to troubleshoot an instance, but you don't necessarily want the instance removed from Octopus when the health check fails.  A better and clenaer approach is to remove the machine from Octopus only when the instance is terminated in AWS.  The following steps will help guide you in creating a Lambda function that will remove machines from Octopus when they are terminated in AWS.

NOTE!  This assumes the name of the machines in Octopus use the AWS Instance ID. This post provides a process to do just that using Chef.  using-chef-to-automate-octopus-deployments

1. Create an IAM role to grant permissions to the Lambda

  Attach to the role one of these managed policies: 
       AWSLambdaVPCAccessExecutionRole (lambda in a VPC)
      AWSLambdaBasicExecutionRole (non VPC lambda)

   Grant the IAM role permissions to decyrpt using KMS and restrict to your KMS key: 
    kms:Decrypt

   Edit the Trust Relationship to allow the lambda to AssumeRole
   {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }

2. Create the Lambda function

Runtime: Node.js 6.10

Code:  Here's a link to the function code, lambda-octopus-cleanup-terminated-machines  copy the code into the Lambda

Environment variables:
  apikey =  Obtain an API key from your Octopus server that has permissions to view and delete machines from Octopus.  Encrypt the api key using the encryption helper on the Lambda setup page
  octopusServer =  <this is the DNS name for your server,  ie:  octopus.yourdomain.com>

Handler:  index.handler

Existing Role:  Select the IAM role created in the previous step

Advanced Settings:
   Timeout:  20 seconds  ( We need allow enough time for the lambda to launch after being idle as instances aren't terminated continuously)

   VPC:  Choose a VPC, Subnets, and Security Groups that will allow your Lambda access to communicate with your Octopus server

KMS Key: the key used to encrypt the Octopus API key

3. Create the trigger to execute the Lambda

To trigger the Lambda we will create a CloudWatch event rule
Event Pattern
Service Name: EC2
Event Type: EC2 Instance State-change Notification
Specific state(s):  Terminated
Any instance

That's it!  Now when an instance is terminated from AWS the Lambda will run and remove the machine from Octopus.