Friday, September 5, 2014

AWS - Auto schedule EC2 instance to start/stop

In order to reduce costs of running test and dev EC2 instances at Amazon I've created a simple script that will run and start or stop an instance based on a schedule.  This script is a PowerShell script that relies on the AWS modules to be installed  ("Import-Module AWSPowerShell")  Refer to the link to Configure PowerShell for AWS  (FYI - configure your default profile using AWS access keys as required)

The script is looking for a tag, 'RunningSchedule' and parsing the value that is set within, which is in a format similar to a cron like style using 24 hour clock : H:H:D or H:H:D-D  (I didn’t include a minutes field)

First 'H' = Start time Hour
Second 'H'  =  Stop time Hour
'D' is for day(s) to run, 1 = Monday, 2=Tues, etc.    So if you want it to run Mon-Fri, enter 1-5
To disable the schedule completely, use ‘Disabled’ for the value.

Example, to have the server start at 8am, stop at 5pm, and run Mon-Fri use:   8:17:1-5
To have the server run from 10am to 3pm on Wed use: 10:15:3

The script will then either start or stop the server based on the schedule.
It will then send an email with the servers listed that were either started or stopped.

To schedule the script itself, I configured a scheduled task in Windows on a server.  Simply run a program "Powershell", path is  'C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe'
Include the optional arguments for the path to your script:  'C:\Scripts\AWS-schedule-start-stop-instances.ps1'


Here is the script:





39 comments:

Anonymous said...

Hi there, thanks for this very helpful script, it's exactly what I was looking for! I am fairly new to AWS but very new to PowerShell, so I apologise for what will no doubt be a very silly question. I was wondering what part needs changing to specify the ID of the instance that I wish to be started?

Do I replaced 'EC2Tag' in Get-EC2Tag with the ID?

Many thanks!

Anonymous said...

Sorry, scrap that. Would changing:

$filter.Name = "resource-type"
$filter.Value = "instance"

to:

$filter.Name = "instance-id"
$filter.Value = "i-xxxx"

Do what I need?

Anonymous said...

Thanks!

You actually do not need to modify anything in the script other than the email parameters.

To specify an instance, you must add an AWS tag to the instance with a name 'RunningSchedule' and the value is the time to start/stop the instance.

I created the script this way to allow developers to control their own servers without affecting the script and possible other servers they don't own.

Thanks

Anonymous said...

Thank you so much for replying. I knew it was a silly question, still getting my head around a lot of this.

Only problem now is that my task scheduler is displaying a last run result of (0x80070001). It has been given highest privileges, so not sure what's wrong with it

Anonymous said...

Are you able to run the Powershell script without the task scheduler? sometimes this can give you an idea of what is failing.

Also, did you configure your machine to run AWS via Powershell?
http://docs.aws.amazon.com/powershell/latest/userguide/pstools-getting-set-up.html

Anonymous said...

Yeh sorry, posted too quickly without thinking. The problem was down to PowerShell as I had assumed it had already been set up for AWS. I'll sort that now. Thanks again for your help. Very much appreciated.

Anonymous said...

Hi. Thanks for sharing this very useful script! I know its been a while since you published. I'm coming from a linux background and not really sure how to go about running the actual script after customization. Could you elaborate on the cmdlets used to actually execute the script?

Ryan Lawyer said...

Hello,
To run the actual script, I configure a scheduled task in Task Scheduler.

Tell it to run a program: 'C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe'
And set optional argument paramater for the path of your script: 'C:\Scripts\AWS-schedule-start-stop-instances.ps1'

If you manually want to run the script, just execute the .ps1 script within Powershell.

Hope that helps

upside said...

For those who are not familiar with with scripts I have written an easy to use windows tool that allows you to schedule starts/stops/backups of EC2 and RDS instances:
http://www.automaticloud.net

Declan Gowran said...

Works Great , Thanks , I have one issue were if I schedule to run every 15 mins it will start stop the instances within the hour between 8-9 starting and 17-18 stopping.

Is the only solution to run every hour ?

Ryan Lawyer said...

Correct I've configured it to run hourly, ignoring the minutes. Part of the reason for this is that Amazon charges by the hour, so if you run the instance for 2 hours and 15 minutes, you will be charged for 3 hours.

Balu Kalepu said...

Great Script.. worked like charm..
Many thanks for sharing the excellent piece.. much appreciated..
Can we also think about the same type of script to take AMIs of the instances on daily or weekly?

Ryan Lawyer said...

Balu,
You could make a script to make an AMI, but there are some things you need to consider. For instance, data integrity when the snapshot is taken. Most ways to handle this is to stop the instance, take the snapshot, then register the snapshot as an AMI.
Are you thinking of doing this for backup purposes? If so, you don't need to create AMI's, just snapshots of the EBS volumes. Though I would recommend 3rd party software for this, as these providers can utilize VSS to ensure data integrity of the volumes.

Thanks
Ryan

Anonymous said...

Hi Ryan. First I want to say thank you for providing such useful scripts and info on your site. I was wondering, currently I have your script running and it runs great if I leave my session signed into the server, but if I logout the scheduled task does not pick up my aws keys info saved under user profile. Is this by design or am I doing something wrong? Also, I am not having luck getting the script to work Mon-Sunday...works fine Mon-Saturday but it doesn't run on Sundays. The RunningSchedule TAG is set as 8:22:1-7 for example to start at 8am, stop at 10pm...works great Mon-Sat...but does not work on Sunday. Any ideas what I can do to fix this? Thanks again.

Ohhh...Anyway I can schedule this task to run and have my aws keys stored encrypted on the server so I do not have to have it in the user profile? Thanks a ton!

Ryan Lawyer said...

I've typically ran the scheduled task as a local system vs user account, there is also an option to run the scheduled task whether logged in or not.

As for the access keys, I'd recommend using IAM roles, if you can run the task from an EC2 instance. The IAM role would need permissions to Get the EC2 tags and instance status.

If that's not possible, you can try placing the keys into the System Environment variables of the server running the task: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Here's a link from AWS with more info: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-environment

I did some testing for Mon-Sun, and it looks like Sunday is represented as '0' instead of '7', so if you want to run it every day, try using '0-6'

Thanks
Ryan

Anonymous said...

How would you set a schedule so that the instance never stops in case someone accidentally shuts it down?
I am looking to set a schedule temporarily on some instances which I don't want to manually restart and I know will be powered off by a job. Will this work?

00:00-0-6

Ryan Lawyer said...

I would take a slightly different approach and simplify the script. Remove the scheduling logic, and instead query a Tag like, "AlwaysOn" and if set to 'True' and the instance's status is stop, then execute the Start-Instance.

Something like this (I haven't tested this, but it should work)

Initialize-awsdefaults

Import-Module AWSPowerShell

#create filter to limit search on instances only
$filter = New-Object amazon.EC2.Model.Filter
$filter.Name = "resource-type"
$filter.Value = "instance"

#create filter based on tag=AlwaysOn
$filterType = New-Object amazon.EC2.Model.Filter
$filterType.Name = "tag:AlwaysOn"
$filterType.Value = "True"

#join the two filters together
$filter = @($filter,$filterType)

#retrieve the instances based on the filter parameters
$instances = Get-EC2Tag -Filter $filter

#cycle through each instance found
foreach($instance in $instances){
#Get current instance state, either Running or Stopped
$state = Get-EC2InstanceStatus -InstanceIds $instance.ResourceId
$state = $state.InstanceState.Name

if ($state -eq "stopped"){
Start-EC2Instance -InstanceIds $instance.ResourceId
}
}

Anonymous said...

Thanks Ryan. Even though it looks like the right code but I tried and it did not start the instances tagged with AlwaysOn to True. It did not give me any error either. It simply executes and comes back to the prompt.

Ryan Lawyer said...

I think this should fix it.
Modify the line to get the state to:
$state = Get-EC2InstanceStatus -InstanceId $instance.ResourceId -IncludeAllInstance $true

Anonymous said...

That worked perfect. Thanks a ton.

Anonymous said...

Hi Ryan,

Thanks for this great scheduler. However I have 2 custom schedules which I am not sure this script can work with?

1. Running Schedule Mon-Fri 7AM to 8PM and Saturday/Sunday (always up)

2. Running schedule Sun-Fri (Always up) and Saturday (shutdown)
24-24:0-5

I am wondering if this script have to be modified to have 2 separate schedules (one for the weekdays and one for the weekend?

Ryan Lawyer said...

Thank you. Unfortunately the scheduler is quite basic in function and therefore would take some modification to allow handling of more complex schedules as you suggest.

One quick way to achieve this is to have different EC2 tags, one for the weekday and another for weekend. Then run two different scripts filtering on the appropriate tags. Not the best approach.

I'll look into adding more complex schedules when/if I find some time.

Anonymous said...

Thanks Ryan. The scheduler is very powerful even in it's very basic form. Appreciate your hard work here. I figured the same that this needs to be modified to take into account custom schedules. Will wait for you work on this when you get time.

I can work with different EC2 tags at the moment and make it work that way. However, I would like to find out how to take into account minutes along with the start and stop time. I understand from AWS billing perspective it's every one hour but from an operational perspective sometimes you need to be granular with your start/stop times.

How do I setup something like this

06:30:20:00:1-5

Thanks in advance.

Ryan Lawyer said...

Adding the minutes to the tag will not work as the format for the parser is expecting HH:HH:D-D. This should be a fairly easy fix. I'll try to get something put together to allow for HH:MM:HH:MM:D-D

Anonymous said...

Thanks for the quick response. Will 24-24:1-5 cover the 24 hour window. I tried and it did not seem to take effect. Basically I am looking for a 24 hour schedule Monday to Friday with Saturday/Sunday off.

Ryan Lawyer said...

For the 24 hour setup, I'd modify the script to only compare to the days of the week, if the current day is part of the tag and the instance is stopped, start it, otherwise do nothing. If the current day is not in the tag and the instance is running, stop it.

As for the minute functionality, I have put together a revised script to add minutes. The EC2 tag must be in the format HH:mm:HH:mm:D-D

NOTE: I have not fully tested this, so please consider testing before putting into a production environment.

Here's a link to GitHub to access it.
https://github.com/SysAdminWaterCooler/AWS/blob/master/EC2-StartStop-HHmm-Scheduler.ps1
Hope this helps!
Ryan

Anonymous said...

Thanks Ryan. I modified the 24 hour setup script below (link included). The issue is now it's breaking the other tags that use HH:HH:D-D.

I set some instances to run Monday to Friday 8 AM to 8 PM with the following tag/value.
RunningSchedule 08:20:1-5.

In the below modified script I want the instance to run 24 hours based on the Days tag only. Therefore I created the following tag/value
RunningSchedule 1-5

Now with this it's starting the instances that were stopped at 8 PM. Any ideas how to fix this? May be I am missing something.


Also, in the HH:mm:HH:mm:D-D script that you uploaded I am getting the error below when I put this tag value
08:00:09:00:1-5

New-TimeSpan : Cannot bind parameter 'End'. Cannot convert value ":" to type "System.DateTime". Error: "String was not
recognized as a valid DateTime."

I am not an expert in this. Your help is appreciated.

24 hour schedule modified script.
https://gist.github.com/vcloudguy/6a456cbf6a834ff23e30d811858dfef9

Anonymous said...

Now I can't run this script at all. It was working for days and weeks and now all of a sudden it has come to a grinding halt. Can you please help? I get this error when I run this scheduler. I using the following tag

RunningSchedule
06:18:1-5

You must provide a value expression on the right-hand side of the '-' operator.
At C:\temp\Scheduler_1.1.ps1:88 char:28
+ if ($now.Hour - <<<< in $stop..$start -and $state -eq "running"){
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : ExpectedValueExpression

Ryan Lawyer said...

For your 24hr schedule, simply use a different tag, say RunningDays and modify this script to query on that tag instead.

Anonymous said...

Thanks Ryan. After running the script with HH:mm:HH:mm:D-D format I can no longer run the original script. Tried changing the tag to RunningDays and query on that but the script just runs and does not do anything at all. Previously my schedules were working fine and now they don't work at all. It seems like the script is just not able to execute any command without throwing any error.

I checked the API calls are working fine with other scripts. Is it possible the HH:mm:HH:mm:D-D script caused some issues?

Ryan Lawyer said...

ah...I believe I know what is happening. The original script is expecting the 'RunningSchedule' tag value to be, HH:HH:D-D. The newer script is expecting format of HH:mm:HH:mm:D-D and is querying on the same tag 'RunningSchedule'.

If you want to use both scripts in your environment, you'll need to change the Tags that you want to use the HH:mm script to something other than 'RunningSchedule'

Hope that helps clarify!

Ryan Lawyer said...

You'll also need to modify the HH:mm script to query the new tag you decide to use.

Ryan Lawyer said...

For the 24hr script, are you using the tag value format of D-D, removing the HH:HH?

If you do this, change the $days parameter to equal $schedule:
$days = $schedule

This should fix your issue.

Ryan

Anonymous said...

Thanks Ryan. I am using different tags in the 24 hour script and the HH:mm:HH:mm:D-D script.

24 hour script- RunningDays
HH:mm:HH:mm:D-D- RunningSchedule

I made the change in the 24 hour script and it worked as expected. But the HH:mm:HH:mm:D-D script stopped executing (same behavior seen as before). It simply executes without any error and does not start/stop instances based on the tag values. I am using the below tags and it does not do anything at all now.

RunningSchedule
08:00:18:00:1-5

I am wondering if the new-timespan cmdlet is causing some issues with the date calculation? Can you try both the scripts together and see if you notice the same behavior?

Anonymous said...

Hi Ryan,

I think I may know what is going on here after doing some more testing. The HH:mm:HH:mm:D-D script has a different start of the week then the previous scripts. In the previous scripts the week starts from 0-6 (Sunday to Saturday) whereas in the HH:mm:HH:mm:D-D script I believe the week starts from 1-7 (Sunday to Saturday). I think that was causing the issue.

May be you can test this when you have a moment. Thanks for your help and efforts. Much appreciated.


Ryan Lawyer said...

I always get a 0 for Sunday. It's using: "Get-Date -Uformat %u" to determine the Day of the week. What version OS/Powershell are you running the script on?

Anonymous said...

Windows 2008 R2 STD

Name Value
---- -----
PSVersion 4.0
WSManStackVersion 3.0
SerializationVersion 1.1.0.1
CLRVersion 4.0.30319.18444
BuildVersion 6.3.9600.16406
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0}
PSRemotingProtocolVersion 2.2


It seems like I cannot run both 24 hour script and the HH:mm:HH:mm:D-D script together. Only one seems to be working fine at a time. I can see you are using Get-Date -Uformat %u but for some reason it's not doing the date calculations correctly. There seems to be some difference between the 2 scripts.

I am reading different tag values in both the scripts.

DAYS- RunningDays
HH:mm- RunningSchedule

It's weird!!

Anonymous said...

Hello,

Thanks a lot for this script!!!

I have one unexpected behavior:
I have an EC2 instance started with Tag Running schedule set to : 8:17:1-2

We are the 07/21 at 1PM ( a Thursday ( day 4 of the week))
So I expect to have this instance stopped if I run the script (because I put 1-2 (Monday to Tuesday )for the days)

But Unfortunately the script didn't stop the instance.
Something can be added to manage this case?




Ryan Lawyer said...

Where you ran the script from, do you have the proper permissions to stop EC2 instances, provided by either access key or IAM roles?

Are you able to stop an instance via Powershell, changing to your instanceID?
Stop-EC2Instance -Instance instanceID

You may also need to set the PowerShell execution policy to UnRestricted:
-ExecutionPolicy unrestricted