AutoScaleFormula Improvements Needed

Question

There is a problem when there is one TVM running with an AutoScaleFormula and the number of JobManager tasks running is equal the MaxTasksPerVM. In this scenario, the TVMs will never scale up.
I have multiple workitems/jobs running tasks in the same pool. Each workitem uses a Job Manager task. The pool is configured to allow 4 MaxTasksPerVM. I use the AutoScaleFormula to scale up/down the TVMs as needed. The problem I am running into,
is that when there are exactly 4 workitems/jobs running at once, each one of them has a job manager running in the single TVM, and the TVMs won't scale up. It just gets stuck at 1 TVM instance filled up with JobManagers, resulting in none of my tasks
running.
We do not have a way to account for the running job managers in the AutoScaleFormula. Here is the formula I am using, which works great when only one workitem/job/jobmanager is running:
lastActiveTaskSample = $ActiveTasks.GetSample(1);
lastRunningTaskSample = $RunningTasks.GetSample(1);
totalTasks = lastActiveTaskSample + lastRunningTaskSample;
$TargetDedicated = min(max(totalTasks / 4, 1), 60);
$TVMDeallocationOption = taskcompletion;
The results of EvaluateAutoScale are:
$TargetDedicated=1;
$TVMDeallocationOption=taskcompletion;
lastActiveTaskSample=[3];
lastRunningTaskSample=[0];
totalTasks=[3]
There are indeed 3 other tasks waiting to run, but there are also 4 JobManager tasks that are running, but aren't being reflected anywhere in the AutoScale results. The documentation doesn't list any variables related to job manager tasks that I can use
in the formula. Ideally, I'd be able to add $JobManagerTasks.GetSample(1) to my formula to make sure the TVMs can still scale up in this scenario.

Answer 1

Okay, we've investigated this issue.  The short answer is that JobManager tasks ARE actually counted in the RunningTasks sample set, but you did hit another bug.
I apologize in advance for the following wall of text...
We strongly recommend you avoid of using "GetSample(1)" in your autoscale formulas.
This is because "GetSample(1)" is just saying "give me the last sample you have, no matter how long ago you got it"  Since you will use these samples to grow/shrink your pool (and your pool costs you money) we recommend that you
base the formula on more than 1 samples worth of data.  Instead we suggest you do some trending type analysis and grow your pool based on that.
You can use the ".GetSample(Interval Lookback Start, Interval Lookback End)" API to get a vector of samples, for example:
runningTasksSample = $RunningTasks.GetSample(60 * TimeInterval_Second, 120 * TimeInterval_Second);
might return:
runningTasksSample=[1,1,1,1,1,1,1,1,1,1,1,1];
Or, if you would like more certainty, you can force the evaluation to fail if there are less than a certain percentage of samples (here percentage means that if in a given time interval there were supposed to be 60 samples, but actually due to networking
failures or other issues we were only able to gather 30 samples, the percentage would be 50%).
This is how you specify a percentage (note the 60 as the 3rd parameter):
runningTasksSample = $RunningTasks.GetSample(60 * TimeInterval_Second, 120 * TimeInterval_Second, 60);
When specifying a time range, always start with the time range starting at least 1m ago, since it takes about 1m for samples to propagate through the system, so samples in the range (0 * TimeInterval_Second, 60 * TimeInterval_Second) will often not be available.
Again you can use the percentage API to force a particular sample percentage.
So instead of using the formula you specified above, we could use something like this instead:
activeTasksSampleVector = $ActiveTasks.GetSample(60 * TimeInterval_Second, 120 * TimeInterval_Second);
runningTasksSampleVector = $RunningTasks.GetSample(60 * TimeInterval_Second, 120 * TimeInterval_Second);
totalTasksVector = activeTasksSampleVector + runningTasksSampleVector;
vmsRequiredVector = totalTasksVector / 4;
vmsRequired = avg(vmsRequiredVector);
$TargetDedicated = min(max(vmsRequired, 1), 60);
Now, the downside of this is that you'll take a bit longer to respond to changes in the running/pending queue length (since you're taking the average), but as it stands right now autoscale doesn't evaluate immediately anyway (it only runs on a pool periodically,
where that period is on the scale of 5-15m).
Autoscale is not (currently) intended as a sub 1m response to changes but rather to adjust the size of your pool gradually as you run a workload. 
Since we only evaluate the formula every ~15m it is not like your pool is going to immediately respond to new task pressure if left to its own devices.  On the other hand if you are running a long lived pool and you want to respond to changes over the
course of a day (for example day/night discrepancies in task load) then autoscale is a good fit.
Why you observed the behavior you did
In addition to the best practices comments I mentioned above, you have hit 2 different bugs.
The last 1-2 samples of $RunningTasks are almost always 0, so ".GetSamples(1)" on RunningTasks often will return 0 even if there are some running tasks.  We will investigate this and work on a fix, but in the meantime adhering to the best
practice of avoiding GetSamples(1) will help you avoid this issue.
This one is more painful -- right now there is a bug where even when MultipleTasksPerVM is not set to the default of 1, $RunningTasks will only report 1 running task (even though there may be up to N, where N == MaxTasksPerVM).  We're already tracking
this bug in our backlog and will get to it ASAP.  In the meantime, you can probably edit your formula to think of $RunningTasks as "RunningVMs" instead -- which should be able to get you close to what you want... see the following formula for
an example of what I mean:
activeTasksSampleVector = $ActiveTasks.GetSample(60 * TimeInterval_Second, 120 * TimeInterval_Second);
runningVMsSampleVector = $RunningTasks.GetSample(60 * TimeInterval_Second, 120 * TimeInterval_Second);
vmsRequiredVector = activeTasksSampleVector / 4 + runningVMsSampleVector;
vmsRequired = avg(vmsRequiredVector);
$TargetDedicated = min(max(vmsRequired, 1), 60);
Thanks and sorry for the inconvenience,
Please let us know if you have any more questions and we'll be happy to help.
-Matt