[Buildbot] #3392: [MOSS Project] Better handling of EC2 to avoid cost overruns

[Buildbot trac]

#3392: [MOSS Project] Better handling of EC2 to avoid cost overruns
---------------------------------------+--------------------
Reporter:  dustin                      |       Owner:
    Type:  enhancement                 |      Status:  new
Priority:  major                       |   Milestone:  0.9.+
 Version:  master                      |  Resolution:
Keywords:  moss, ec2cost, ec2, bounty  |
---------------------------------------+--------------------
Description changed by dustin:

Old description:

> This involves shutting down the instances, particularly when the master
> is stopped and/or crashes.  This will help users keep their EC2 costs
> contained.
>
> Users should be able to rely on Buildbot to only spend the necesary
> amount on EC2 instances, without any surprises at the end of the month.
> Successful completion of this project will address the known bugs below,
> but also provide some failsafe or monitoring mechanisms that an OSS
> project could use to head off any overbilling.
>
> == relevant bugs ==
> [[TicketQuery(order=priority,status!=closed,keywords~=ec2cost,format=table,col=summary|owner)]]
>
> Bounty: US$5,000 - see BountyProgram

New description:

 This involves shutting down the instances, particularly when the master is
 stopped and/or crashes.  This will help users keep their EC2 costs
 contained.

 Users should be able to rely on Buildbot to only spend the necesary amount
 on EC2 instances, without any surprises at the end of the month.
 Successful completion of this project will address the known bugs below,
 but also provide some failsafe or monitoring mechanisms that an OSS
 project could use to head off any overbilling.

 == project scope ==

 Completing this project will involve:

  * Establishing and documenting a recommended way to run Buildbot in EC2,
 encompassing worker configuration and startup
  * Providing support for shutting down an instance when it cannot connect
 to the master for a prolonged period (#3393)
  * Correcting spot-instance handling so that instances are not "lost" by
 the buildbot master (#2935)
  * Providing a failsafe method of managing EC2 instances that will
 reliably prevent over-provisioning and lost EC2 instances.  Proposals
 should outline one or more specific approaches to this problem.

 == bounty ==

 Bounty: US$5,000 - see BountyProgram

--

--
Ticket URL: <http://trac.buildbot.net/ticket/3392#comment:7>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation