Skip to content

[DurableTask.ServiceBus] Fix blob container listing and purge for hub names with dashes and culture-dependent container names - #1424

Open
Davide Montanari (davidemontanari) wants to merge 3 commits into
Azure:mainfrom
davidemontanari:davidemontanari/dtfx-sb-fix-blob-container-purge-invariant-culture-name
Open

Davide Montanari (davidemontanari) wants to merge 3 commits into
Azure:mainfrom
davidemontanari:davidemontanari/dtfx-sb-fix-blob-container-purge-invariant-culture-name

Conversation

@davidemontanari

@davidemontanari Davide Montanari (davidemontanari) commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes how DurableTask.ServiceBus names, lists and purges overflow blob containers, so that:

  • purge works for hub names with dashes;
  • each hub only lists and deletes its own, correctly named containers;
  • container names don't depend on the host culture.

Also includes two small test-related cleanups.

Fixes

Expired containers were never purged for hub names with dashes

Overflow containers are named {hubName}-dtfx-{type}-{yyyyMMdd}. BlobStorageClientHelper.IsContainerExpired required exactly 4 dash-separated segments, so for a hub like my-hub every container was skipped. PurgeOrchestrationHistoryAsync never deleted its overflow blobs, and storage grew without limit.

The check now reads the fixed segments from the end and requires the dtfx marker, so any hub name works.

Container listing included other hubs' and malformed containers

BlobStorageClient.ListContainersAsync filtered by the {hubName}-dtfx prefix only. Because hub names can contain dashes, this returned:

  • other hubs' containers: for hub alpha, the listing included alpha-dtfx-backup-dtfx-session-20150516, which belongs to hub alpha-dtfx-backup;
  • malformed containers: names such as alpha-dtfx-session-backup or alpha-dtfx-session-20150230.

Deleting a hub (DeleteBlobStoreContainersAsync) therefore also deleted these, including another hub's overflow messages and session state.

ListContainersAsync now returns only containers owned by this hub (new internal BlobStorageClientHelper.IsContainerOwnedByHub):

  • the name without its trailing -{type}-{date} must equal this hub's {hubName}-dtfx prefix;
  • the type segment must not be empty;
  • the last segment must be a valid yyyyMMdd calendar date (invariant culture).

Both DeleteExpiredContainersAsync and DeleteBlobStoreContainersAsync use this listing.

Container names depended on the host culture

  • Dates: written with the current culture. On hosts using a non-Gregorian calendar (e.g. th-TH, where 2015 is 2558), containers got far-future dates. The expiry check parses dates with the invariant culture, so these containers never expired. Dates are now written with CultureInfo.InvariantCulture. Blob reads and writes are not affected, because the blob key carries the exact container name.
  • Lower-casing: done with ToLower(). On tr-TR, I becomes a dotless ı, which is not valid in a container name. Now uses ToLowerInvariant(), and a redundant ToLower() in BlobStorageClient is removed.

Container name suffix validation accepted invalid characters

The regex @"^[a-z0-9\\-]+$" also matched a backslash (\\ in a verbatim string is a literal backslash), and $ matched before a trailing \n. It is now ^[a-z0-9-]+\z, which accepts only lowercase letters, digits and dashes. It is built once as a static field.

Cleanups

  • ServiceBusOrchestrationService.GetHubQueueMaxDeliveryCountsAsync (internal, used only by tests): Single → SingleOrDefault, so a missing queue returns the intended -1 instead of throwing.
  • WaitForOrchestrationTests no longer uses a fake connection string containing SharedAccessKey=..., which could trigger secret scanners. It now uses a new shared NoOpTokenCredential test helper with the host-name + TokenCredential constructor.

Behaviour changes / upgrade notes

  • BlobStorageClient.ListContainersAsync (public) no longer returns:
    • other hubs' containers that share the prefix;
    • containers outside the {hubName}-dtfx-{type}-{yyyyMMdd} format. The framework never creates these, but any created manually are no longer listed or deleted with the hub.
  • Purging is on demand. It only runs when the application calls TaskHubClient.PurgeOrchestrationHistoryAsync. For hubs whose names contain dashes, the first purge after upgrading deletes the backlog of the hub's own overflow containers older than the threshold, which may be a large one-off batch.
  • Choosing a threshold: containers are judged by the date in their name. Choose a threshold longer than your longest-idle orchestration, so session state that overflowed long ago is not purged.
  • Old non-Gregorian containers: containers already created with non-Gregorian dates (e.g. year 2558) are not purged automatically and need a one-time manual cleanup.
  • Scope: hub names with dashes apply to the blob store only. Azure Table names don't allow dashes, so AzureTableInstanceStore needs a dash-free hub name; that is unchanged.

Tests

  • BlobStorageClientHelperTest, offline:
    • IsContainerExpiredHubNameWithDashesTest
    • ContainerNamesAreCultureInvariantTest (th-TH, tr-TR)
    • ParseKeyRejectsInvalidSuffixCharactersTest (backslash, trailing newline)
    • IsContainerOwnedByHubTest (overlapping hubs, dashed hubs, empty type, non-date suffixes, invalid calendar dates)
  • BlobStorageClientTest, storage-backed:
    • TestDeleteExpiredContainersPreservesOverlappingHub: with hubs alpha and alpha-dtfx-backup in one account, purging alpha deletes only its own expired container; the other hub's containers remain and its session blob is still readable.
    • TestDeleteBlobStoreContainersPreservesOverlappingHub: alpha lists only its own containers, and deleting the hub keeps the other hub's containers.
    • TestDeleteBlobStoreContainersIgnoresMalformedContainers: {prefix}-session-backup and {prefix}-session-20150230 are neither listed nor deleted with the hub.
  • Each new test fails against the code without the corresponding fix.
  • 44 tests pass on net8.0 and net48:
    • BlobStorageClientHelperTest
    • BlobStorageClientTest (run against Azurite)
    • ServiceBusUtilsTests
    • WaitForOrchestrationTests

Copilot AI balanced review requested due to automatic review settings October 22, 2026 17:21
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Purging a hub can delete another hub’s overflow containers when their naming prefixes overlap.

1 open issue
What changed in this PR

This PR fixes Service Bus overflow blob naming and expiration handling for dashed hub names and culture-dependent formatting, with regression tests and small test utilities cleanups.

Changes:

  • Parse expiration suffixes from the end and use invariant date formatting and lowercasing.
  • Tighten suffix validation and add regression tests for dashed hubs, cultures, and invalid characters.
  • Return -1 for missing queues and replace fake test credentials with a token credential stub.
File Description
test/​DurableTask.ServiceBus.Tests/​WaitForOrchestrationTests.cs Replace the fake connection string with a credential stub.
test/​DurableTask.ServiceBus.Tests/​NoOpTokenCredential.cs Add a credential stub for offline tests.
test/​DurableTask.ServiceBus.Tests/​BlobStorageClientHelperTest.cs Add naming, expiration, and suffix validation tests.
src/​DurableTask.ServiceBus/​Tracking/​BlobStorageClientHelper.cs Fix dashed-name parsing, invariant formatting, and suffix validation.
src/​DurableTask.ServiceBus/​Tracking/​BlobStorageClient.cs Delegate lowercasing to the invariant helper.
src/​DurableTask.ServiceBus/​ServiceBusOrchestrationService.cs Handle missing queues in delivery-count lookup.

Review effort: Balanced

Comment thread src/DurableTask.ServiceBus/Tracking/BlobStorageClientHelper.cs
@davidemontanari Davide Montanari (davidemontanari) changed the title [DurableTask.ServiceBus] Fix blob container purge for hub names with dashes and culture-dependent container names [DurableTask.ServiceBus] Fix blob container listing and purge for hub names with dashes and culture-dependent container names Oct 22, 2026
Copilot AI balanced review requested due to automatic review settings October 22, 2026 18:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The ownership filter still allows malformed-date containers to be deleted despite the documented exclusion guarantee.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread src/DurableTask.ServiceBus/Tracking/BlobStorageClientHelper.cs Outdated
Copilot AI balanced review requested due to automatic review settings October 22, 2026 18:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review identified no blocking issues in the focused fixes and regression coverage; reported test results were not independently verified.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants