Query Azure Table using PowerShell and REST API with Pagination

Update 2020-09-30: Apparently MS removed x-ms-continuation-NextRowKey and x-ms-continuation-NextPartitionKey from the REST API, even though it is still documented, so this code no longer works.

###CODE DOES NOT WORK####
$sasUri = "https://xxxx.table.core.windows.net/ExampleTable?[insert SAS key here]"

$tableData = New-Object -TypeName "System.Collections.ArrayList"
$rowKey = $null
$partitionId = $null

do {
    $i++
    $headers = @{
        "x-ms-date"=(Get-Date -Format r);
        "x-ms-version"="2016-05-31";
        "Accept-Charset"="UTF-8";
        "DataServiceVersion"="3.0;NetFx";
        "MaxDataServiceVersion"="3.0;NetFx";
        "Accept"="application/json;odata=nometadata"
    }

    $response = Invoke-WebRequest -Method Get -Uri $sasUri -Headers $headers -UseBasicParsing
    $workingTable = ($response.Content | ConvertFrom-Json).value
    $workingTable | % {$tableData.Add($_) | Out-Null}
    
    $rowKey = $response.Headers.'x-ms-continuation-NextRowKey'
    $partitionId = $response.Headers.'x-ms-continuation-NextPartitionKey'
    $sasUri = $sasUri + "&NextPartitionKey=$partitionId&NextRowKey=$rowKey"
} while ($workingTable.Count -eq 1000)

Upload to Azure Tables using PowerShell/REST with Programmatically Updated Row Numbers (now with Pagination!)

Update 2020-12-23: It appears I was partially correct. The pagination weren’t removed from MS, but it seems if you delete a record from the table, the keys disappear.

Update 2020-09-30: Apparently MS removed x-ms-continuation-NextRowKey and x-ms-continuation-NextPartitionKey from the REST API, even though it is still documented, so this code no longer works.

Update 2020-08-03: I found that there is a 1000 record limit when querying Azure Tables. Therefore the code block determining row key has been rewritten. For convenience’s sake, my next article will contain just the code needed to query an Azure table with pagination.

####CODE DOES NOT WORK####
#with thanks to http://blog.tofte-it.dk/powershell-azure-storage-using-rest-api/
$sasUri = "https://xxxx.table.core.windows.net/ExampleTable?[insert SAS key here]" #need Query+Add permissions
$workingSasUri = $sasUri

$tableData = New-Object -TypeName "System.Collections.ArrayList"
$rowKey = $null
$partitionId = $null

do {
    $i++
    $headers = @{
        "x-ms-date"=(Get-Date -Format r);
        "x-ms-version"="2016-05-31";
        "Accept-Charset"="UTF-8";
        "DataServiceVersion"="3.0;NetFx";
        "MaxDataServiceVersion"="3.0;NetFx";
        "Accept"="application/json;odata=nometadata"
    }

    $response = Invoke-WebRequest -Method Get -Uri $workingSasUri -Headers $headers -UseBasicParsing
    $workingTable = ($response.Content | ConvertFrom-Json).value
    $workingTable | % {$tableData.Add($_) | Out-Null}
    
    $rowKey = $response.Headers.'x-ms-continuation-NextRowKey'
    $partitionId = $response.Headers.'x-ms-continuation-NextPartitionKey'
    $workingSasUri = $workingSasUri + "&NextPartitionKey=$partitionId&NextRowKey=$rowKey"
} while ($workingTable.Count -eq 1000)

$rowArray = $tableData
$rowArray = New-Object -TypeName "System.Collections.ArrayList"
$tableData.RowKey | % {$rowArray.Add([int]$_) | Out-Null}
$rowKey = $rowArray | sort | select -Last 1
if($rowKey){$rowKey += 1}
else {$rowKey = 1}

$tableData = @{
    "PartitionKey" = "1";
    "RowKey" = $rowKey.ToString();
    "dataYouWant1" = "foo";
    "dataYouWant2" = "bar";
    "etc." = "etc."
}

$headers = @{
    "x-ms-date"=(Get-Date -Format r);
    "x-ms-version"="2016-05-31";
    "Accept-Charset"="UTF-8";
    "DataServiceVersion"="3.0;NetFx";
    "MaxDataServiceVersion"="3.0;NetFx";
    "Accept"="application/json;odata=nometadata"
}

$body = [System.Text.Encoding]::UTF8.GetBytes((ConvertTo-Json -InputObject $tableData))
Invoke-WebRequest -Method Post -Uri $sasUri -Headers $headers -Body $body -ContentType "application/json" -UseBasicParsing