Taking packet captures on the Teams Connector

This procedure describes how to take packet captures on the Teams Connector. There is a manual and a scripted process.

Manual process

Setting the instance count to 1 is optional but if you skip this step you will have to capture on each Teams Connector instance simultaneously.

Log in to the Azure portal and go to the main resource group (<prefix>-TeamsConn-<region>-RG).
Drill down into the virtual machine scale set, select Scaling, take a note of the current number and set the Instance count to 1, then wait for this process to complete (it can take up to 30 minutes).
When complete, connect via RDP onto the Teams Connector.
Launch a PowerShell session as administrator.
Run the following command (update the path if you have specified an alternative username):

netsh trace start scenario=NetConnection capture=yes report=no persistent=no maxsize=2048 correlation=yes traceFile=c:\users\pexadmin\desktop\netTrace_$(get-date -Format yyyyMMdd_HHmm).etl

Note that maxsize=1024 is likely wrap quickly on busy deployments and it should be adjusted to a higher value (e.g. 3072) if necessary.
When all of your captures are running (usually Pexip Proxying Edge Nodes and Teams Connector captures), replicate a call with your issue.
Stop the capture on the Teams Connector by running netsh trace stop in the same PowerShell session.
Gather the captures from the Teams Connector and the Pexip nodes.
Upload the captures along with a new snapshot from Pexip and upload these to https://upload.pexip.com/ inputting your ticket number.
Take a note of the callid and send it across via the ticket.
You can now set the instance count back to the previous number (if it was larger than 1):
1. Login to the Azure portal and go to the main resource group (<prefix>-TeamsConn-<region>-RG), drill down into the virtual machine scale set, select Scaling and set the Instance count to the previous number.
2. Wait for this process to complete (it can take up to 30 minutes).

Scripted process

This fully-scripted process can capture on multiple instances in a scale set.

Requirements

To use this process you need:

Azure CLI — https://docs.microsoft.com/en-us/cli/azure/install-azure-cli
Variables script from your Teams Connector installation
StartPacketCapture.ps1 (see below)
StopPacketCapture.ps1 (see below)

Process

Save all of the required base scripts in a local directory.
Launch PowerShell ISE.
Change directory to the location with the required base scripts (from step 1).
Load the variables script from your Teams Connector installation into your current PowerShell ISE session.

Run through the script below in a stepped manner using PowerShell ISE, ensure that the required scripts are in the correct folder.

Copy to clipboard

# Capture filename (located on the connector instance)
$CaptureFile = "P:\netTrace_$(get-date -Format yyyyMMdd_HHmm).etl"
$ScriptRoot = Get-Location

# Login
az login

# Set subscription
az account set --subscription $PxSubscriptionId

# Get instances
$GetVMSSName = az vmss list --resource-group $PxTeamsConnResourceGroupName | ConvertFrom-Json
$SetVMSSName = $GetVMSSName.name
$Instances = az vmss list-instances -n $SetVMSSName -g $PxTeamsConnResourceGroupName --output tsv --query '[*].{instanceId:instanceId}'

# Start captures
foreach ($i in $Instances) {
    az vmss run-command invoke --command-id RunPowerShellScript --instance-id $i -n $SetVMSSName -g $PxTeamsConnResourceGroupName --scripts @$ScriptRoot\StartPacketCapture.ps1 --parameters "TraceFilePath=${CaptureFile}"
}

# Stop captures
foreach ($i in $Instances) {
    az vmss run-command invoke --command-id RunPowerShellScript --instance-id $i -n $SetVMSSName -g $PxTeamsConnResourceGroupName --scripts @$ScriptRoot\StopPacketCapture.ps1
}

RDP to each host, compress the netTrace files on the P:\ drive and upload them via the link your support engineer has provided.

Required scripts

StartPacketCapture.ps1

Copy to clipboard

<#    
Author: https://github.com/adbertram/Random-PowerShell-Work/blob/master/Networking/PacketCapture.ps1
.SYNOPSIS
    This function starts a packet trace using netsh. Upon completion, it will begin capture all
    packets coming into and leaving the local computer and will continue to do do until
    Stop-PacketCapture is executed.
.EXAMPLE
    PS> Start-PacketTrace -TraceFilePath C:\Tracefile.etl

        This example will begin a packet capture on the local computer and place all activity
        in the ETL file C:\Tracefile.etl.
    
.PARAMETER TraceFilePath
    The file path where the trace file will be placed and recorded to. This file must be an ETL file.
        
.PARAMETER Force
    Use the Force parameter to overwrite the trace file if one exists already
    
.INPUTS
    None. You cannot pipe objects to Start-PacketTrace.

.OUTPUTS
    None. Start-PacketTrace returns no output upon success.
#>
[CmdletBinding()]
[OutputType()]
param
(
    [Parameter()]
    [ValidateNotNullOrEmpty()]
    [ValidateScript({ Test-Path -Path ($_ | Split-Path -Parent) -PathType Container })]
    [ValidatePattern('.*\.etl$')]
    [string[]]$TraceFilePath,
    [Parameter()]
    [switch]$Force
)
begin {
    Set-StrictMode -Version Latest
    $ErrorActionPreference = [System.Management.Automation.ActionPreference]::Stop
}
process {
    try {
        if (Test-Path -Path $TraceFilePath -PathType Leaf) {
            if (-not ($Force.IsPresent)) {
                throw "An existing trace file was found at [$($TraceFilePath)] and -Force was not used. Exiting.."
            } else {
                Remove-Item -Path $TraceFilePath
            }
        }
        $Process = Start-Process "$($env:windir)\System32\netsh.exe" -ArgumentList "trace start scenario=NetConnection capture=yes report=no persistent=no maxsize=2048 correlation=yes tracefile=$TraceFilePath" -RedirectStandardOutput ".\NUL" -Wait -NoNewWindow -PassThru
        if ($Process.ExitCode -notin @(0, 3010)) {
            throw "Failed to start the packet trace. Netsh exited with an exit code [$($Process.ExitCode)]"
        } else {
            Write-Verbose -Message "Successfully started netsh packet capture. Capturing all activity to [$($TraceFilePath)]"
        }
        catch {
        Write-Error -Message "$($_.Exception.Message) - Line Number: $($_.InvocationInfo.ScriptLineNumber)"
    }
}

StopPacketCapture.ps1

Copy to clipboard

<# 
Author: https://github.com/adbertram/Random-PowerShell-Work/blob/master/Networking/PacketCapture.ps1
.SYNOPSIS
    This function stops a packet trace that is currently running using netsh.
.EXAMPLE
    PS> Stop-PacketTrace

        This example stops any running netsh packet capture.    
.INPUTS
    None. You cannot pipe objects to Stop-PacketTrace.

.OUTPUTS
    None. Stop-PacketTrace returns no output upon success.
#>
[CmdletBinding()]
[OutputType()]
param
()
begin {
    Set-StrictMode -Version Latest
    $ErrorActionPreference = [System.Management.Automation.ActionPreference]::Stop
}
process {
    try {
        $Process = Start-Process "$($env:windir)\System32\netsh.exe" -ArgumentList 'trace stop' -Wait -NoNewWindow -PassThru -RedirectStandardOutput ".\NUL"
    } catch {
        Write-Error -Message "Error: $($_.Exception.Message) - Line Number: $($_.InvocationInfo.ScriptLineNumber)"
    }
}