Capturing log files from multiple .ps1 scripts called from within a .bat file

Question

I am trying to invoke multiple instances of a powershell script and capture individual log files from each of them. I can start the multiple instances by calling 'start powershell' several times, but am unable to capture logging. If I use 'call powershell'
I can capture the log files, but the batch file won't continue until that current 'call powershell' has completed.
ie.  within Test.bat
start powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > a.log 2>&1
timeout /t 60
start powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > b.log 2>&1
timeout /t 60
start powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > c.log 2>&1
timeout /t 60
start powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > d.log 2>&1
timeout /t 60
start powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > e.log 2>&1
timeout /t 60
start powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > f.log 2>&1
the log files get created but are empty.  If I invoke 'call' instead of start I get the log data, but I need them to run in parallel, not sequentially.
call powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > a.log 2>&1
timeout /t 60
call powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > b.log 2>&1
timeout /t 60
call powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > c.log 2>&1
timeout /t 60
call powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > d.log 2>&1
timeout /t 60call powershell . \Automation.ps1 %1 %2 %3 %4 %5 %6 > e.log 2>&1
Any suggestions of how to get this to work?

Answer 1

Batch files are sequential by design (batch up a bunch of statements and execute them). Call doesn't run in a different process, so when you use it the batch file waits for it to exit. From CALL:
Calls one batch program from another without stopping the parent batch program
I was hoping for the documentation to say the batch file waits for CALL to return, but this is as close as it gets.
Start(.exe), "Starts a separate window to run a specified program or command". The reason it runs in parallel is once it starts the target application start.exe ends and the batch file continues. It has no idea about the powershell.exe process
that you kicked off. Because of this reason, you can't pipe the output.
Update: I was wrong, you can totally redirect the output of what you run with start.exe.
How about instead of running a batch file you run a PowerShell script? You can run script blocks or call individual scripts in parallel with the
Start-Job cmdlet.
You can monitor the jobs and when they complete, pipe them to
Receive-Job to see their output. 
For example:
$sb = {
Write-Output "Hello"
Sleep -seconds 10
Write-Output "Goodbye"
Start-Job -Scriptblock $sb
Start-Job -Scriptblock $sb
Here's a script that runs the scriptblock $sb. The script block outputs the text "Hello", waits for 10 seconds, and then outputs the text "Goodbye"
Then it starts two jobs (in this case I'm running the same script block)
When you run this you receive this for output:
PS> $sb = {
>> Write-Output "Hello"
>> Sleep -Seconds 10
>> Write-Output "Goodbye"
>> }
>>
PS> Start-Job -Scriptblock $sb
Id Name State HasMoreData Location Command
1 Job1 Running True localhost ...
PS> Start-Job -Scriptblock $sb
Id Name State HasMoreData Location Command
3 Job3 Running True localhost ...
PS>
When you run Start-Job it will execute your script or scriptblock in a new process and continue to the next line in the script.
You can see the jobs with
Get-Job:
PS> Get-Job
Id Name State HasMoreData Location Command
1 Job1 Running True localhost ...
3 Job3 Running True localhost ...
OK, that's great. But we need to know when the job's done. The Job's Status property will tell us this (we're looking for a status of "Completed"), we can build a loop and check:
$Completed = $false
while (!$Completed) {
# get all the jobs that haven't yet completed
$jobs = Get-Job | where {$_.State.ToString() -ne "Completed"} # if Get-Job doesn't return any jobs (i.e. they are all completed)
if ($jobs -eq $null) {
$Completed=$true
} # otherwise update the screen
else {
Write-Output "Waiting for $($jobs.Count) jobs"
sleep -s 1
This will output something like this:
Waiting for 2 jobs
Waiting for 2 jobs
Waiting for 2 jobs
Waiting for 2 jobs
Waiting for 2 jobs
Waiting for 2 jobs
Waiting for 2 jobs
Waiting for 2 jobs
Waiting for 2 jobs
Waiting for 2 jobs
When it's done, we can see the jobs have completed:
PS> Get-Job
Id Name State HasMoreData Location Command
1 Job1 Completed True localhost ...
3 Job3 Completed True localhost ...
PS>
Now at this point we could pipe the jobs to Receive-Job:
PS> Get-Job | Receive-Job
Hello
Goodbye
Hello
Goodbye
PS>
But as you can see it's not obvious which script is which. In your real scripts you could include some identifiers to distinguish them.
Another way would be to grab the output of each job one at a time:
foreach ($job in $jobs) {
$job | Receive-Job
If you store the output in a variable or save to a log file with Out-File. The trick is matching up the jobs to the output. Something like this may work:
$a_sb = {
Write-Output "Hello A"
Sleep -Seconds 10
Write-Output "Goodbye A"
$b_sb = {
Write-Output "Hello B"
Sleep -Seconds 5
Write-Output "Goodbye B"
$job = Start-Job -Scriptblock $a_sb
$a_log = $job.Name
$job = Start-Job -Scriptblock $b_sb
$b_log = $job.Name
$Completed = $false
while (!$Completed) {
$jobs = Get-Job | where {$_.State.ToString() -ne "Completed"}
if ($jobs -eq $null) {
$Completed=$true
else {
Write-Output "Waiting for $($jobs.Count) jobs"
sleep -s 1
Get-Job | where {$_.Name -eq $a_log} | Receive-Job | Out-File .\a.log
Get-Job | where {$_.Name -eq $b_log} | Receive-Job | Out-File .\b.log
If you check out the folder you'll see the log files, and they contain the script contents:
PS> dir *.log
Directory: C:\Users\jwarren
Mode LastWriteTime Length Name
-a--- 1/15/2014 7:53 PM 42 a.log
-a--- 1/15/2014 7:53 PM 42 b.log
PS> Get-Content .\a.log
Hello A
Goodbye A
PS> Get-Content .\b.log
Hello B
Goodbye B
PS>
The trouble though is you won't get a log file until the job has completed. If you use your log files to monitor progress this may not be suitable.
Jason Warren
@jaspnwarren
jasonwarren.ca
habaneroconsulting.com/Insights