How to Run Commands in Parallel in PowerShell

1. the Windows PowerShell Multithreading
2. Parallel Execution of Scripts With PSJobs
3. Keep Track of Our Jobs
4. Retrieve Job Output
5. Create Scheduled Jobs

The default PowerShell session is single-threaded. It runs one command and moves to the following command.

This method is excellent since it keeps everything repeatable and doesn’t use many resources. In that case, it is time to start thinking about multithreading.

This article will teach us to understand and use various PowerShell multithreading techniques to process multiple data streams simultaneously but managed through the same console.

the Windows PowerShell Multithreading

Multithreading is a method to run more than one command at a time. PowerShell usually uses a single thread; there are many ways to use more than one to parallelize our script.

The primary purpose of multithreading is to decrease the runtime of the code. This time decrease is at the tradeoff of a higher processing power requirement.

In addition, many actions are being performed at once when multithreading, thus requiring more system resources.

Note that we will not see perfect scaling. Spinning up and tearing down items in the script will take some time.

PowerShell needs to run the code using a single thread, and it’s done. We will use the original thread used to run our console to manage the other threads with multiple threads.

That original thread will be maxed out at a certain point, keeping all of the other threads in line.

Parallel Execution of Scripts With PSJobs

One of the most convenient ways to multithread a script is with PSJobs. PSJobs have cmdlets built into the Microsoft.PowerShell.Core module.

The Microsoft.PowerShell.Core module has been included in all versions of PowerShell since version 3. Commands in this module allow us to run code in the background while continuing to run different code in the foreground.

Get-Command *-Job

Keep Track of Our Jobs

Below we will find a list of the most common states a job can be.

1. Completed – The job has finished, and we can retrieve the output data or remove the job.
2. Running – The job is running and cannot be removed without stopping the job forcefully. Output cannot be retrieved.
3. Blocked – The job is still executing, but the user is being prompted for additional information before it can proceed with the execution.
4. Failed – A thrown terminating error occurred while the job ran.

We use the Get-Job command to get the job status started and all of the attributes of our jobs. The output for a job where we can see the state is Running.

The example below executes the Start-Sleep 5 code within an appointment using the Start-Job command. The status of that job is then returned using the Get-Job command.

Example Code:

Start-Job -ScriptBlock { Start-Sleep 5 }
Get-Job

Output:

Id     Name            PSJobTypeName   State         HasMoreData     Location             Command
--     ----            -------------   -----         -----------     --------             -------
1      Job1            BackgroundJob   Running       True            localhost            Start-Sleep 5

Retrieve Job Output

Sometimes the code inside the job returns the output. We can retrieve that code using the Receive-Job command.

It accepts PSJob as the input and then writes the job’s work to the console. Anything that resulted from the job while running has been stored.

When the job is retrieved, it displays all of what was stored.

An example of this would be executing the script below. This script will create and start a job to write the Hello World string to the output.

It then fetches the output from the job and displays it to the console.

Example code:

$Job = Start-Job -ScriptBlock { Write-Output 'Hello World' }
Receive-Job $Job

Output:

Hello World

Create Scheduled Jobs

Another way to interact with PSJobs is through a scheduled job. It’s similar to a Windows scheduled task that we can configure with Task Scheduler.

Scheduled appointments create a way to easily schedule complex PowerShell script blocks in a scheduled task. Then, we can run a PSJob in the background based on triggers.

Job Triggers

Job triggers can be a specific time when a user logs on, when the system boots, and many others. We can also have the triggers repeat at intervals.

These said triggers are defined with the New-JobTrigger command, which specifies a trigger that will execute the scheduled job.

In addition to having a PSJob trigger, we would still have a script block like what is used with a regular PSJob. Once we have both the trigger and script block, we would use the Register-ScheduledJob command to create the job, as shown in the next section.

This command specifies attributes of the scheduled job like the script block that will be run and triggers created with the New-JobTrigger command.

Perhaps we need some PowerShell code to run every time someone logs into a computer. We can create a scheduled job for this.

We would first define a trigger using New-JobTrigger and specify the scheduled job as indicated below. This scheduled job below will write a line to a log file every time someone logs in to the system.

Example code:

$Trigger = New-JobTrigger -AtLogon
$Script = { "User $env:USERNAME logged at $(Get-Date -Format 'y-M-d H:mm:ss')" | Out-File -FilePath C:\Temp\User_Login.log -Append }

Register-ScheduledJob -Name Login_Log -ScriptBlock $Script -Trigger $Trigger

Output:

Id         Name            JobTriggers     Command                                  Enabled   
--         ----            -----------     -------                                  -------   
1          Login_Log       1               "User $env:USERNAME logged at $(Ge... True

Once we run the above commands, we will get a result similar to when doing a new job that will display the ID, the script block, and some other attributes.

Use the AsJob Parameter

Another use of jobs is the -AsJob parameter built into many PowerShell commands. Since there are many different commands, we can find all of them using Get-Command.

Get-Command -ParameterName AsJob

One of the most prevalent cmdlets is Invoke-Command. Usually, when we run this command, it will start executing a command immediately.

However, while some commands will immediately return, allowing us to continue with what we were doing, some will wait until the cmdlet is finished.

While we can use the AsJob parameter with the local machine most of the time, the Invoke-Command cmdlet does not have a native option to run on the local machine. However, there is a workaround by using Localhost as the ComputerName parameter value.

Example:

Invoke-Command -ScriptBlock { Start-Sleep 5 } -ComputerName localhost

The below script uses the Invoke-Command cmdlet to sleep for five seconds and then repeats the same command using the AsJob parameter to show the difference in execution times.

Example code:

Measure-Command { Invoke-Command -ScriptBlock { Start-Sleep 5 } }
Measure-Command { Invoke-Command -ScriptBlock { Start-Sleep 5 } -AsJob -ComputerName localhost }

Output:

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 5
Milliseconds      : 20
Ticks             : 50206571
TotalDays         : 5.81094571759259E-05
TotalHours        : 0.00139462697222222
TotalMinutes      : 0.0836776183333333
TotalSeconds      : 5.0206571
TotalMilliseconds : 5020.6571

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 36
Ticks             : 365754
TotalDays         : 4.23326388888889E-07
TotalHours        : 1.01598333333333E-05
TotalMinutes      : 0.00060959
TotalSeconds      : 0.0365754
TotalMilliseconds : 36.5754