Pull request

Summary

Related to #12436 ("Which properties are required for Microsoft.Testing.Extensions.TrxReport?"). This PR does not close it; see "Out of scope" below.

Problem. A TestNode that has a final state but neither TrxFullyQualifiedTypeNameProperty nor TestMethodIdentifierProperty makes TrxReportEngine.GetClassAndMethodName throw InvalidOperationException ("The test framework '...' with UID '...' does not support the 'ITrxReportCapability' leading to missing or incomplete TRX reports"). That is the shape reported by frameworks without source or type information (the issue notes xUnit.net supports test cases that don't come from source code at all). With the released 2.5.3 package the exception escapes report generation: the test host process crashes, the test application exits with code 7 and the TRX contains no test results (reproduced below).

Change. When neither property is present, the class name is now the documented fallback DefaultClassName, and the method name is the node's display name (already the behavior of the caller, testMethodName ?? testResultDisplayName).

• TrxReportEngine.cs / TrxReportEngine.Results.cs: new private const string FallbackClassName = "DefaultClassName". GetClassAndMethodName returns (FallbackClassName, null) instead of throwing when there is no method identifier (and no TrxFullyQualifiedTypeNameProperty). The method no longer uses instance state, so it became static, and the now-unused resources using was removed. The TrxReportFrameworkDoesNotSupportTrxReportCapability resource is still used by TrxDataConsumer for the "capability not supported" warning, so there are no .resx/.xlf changes.
• Unchanged: nodes that carry either property (TrxFullyQualifiedTypeNameProperty still wins, then namespace + type name from the method identifier); the exceptions for a missing final state and for duplicate singleton properties.
• Docs: the ITrxReportCapability XML documentation and the Microsoft.Testing.Extensions.TrxReport.Abstractions package readme (PACKAGE.md) now say which properties are required (the final state), recommended (TrxFullyQualifiedTypeNameProperty or TestMethodIdentifierProperty, with this fallback), optional (everything else), and that duplicates of a singleton property make report generation fail. This extends the documentation added in Document required and optional TRX report metadata #12458 and removes its now-stale sentence "Omitting both identity properties causes report generation to fail."
• Tests: 4 new test cases in the existing TRX unit tests (see "Test-value evidence").
• No public API change: PublicAPI.*.txt, resources and global.json are untouched. No changelog entry: docs/Changelog-Platform.md is updated in bulk at release time and recent TRX fix PRs don't touch it.

Out of scope (why this is "Related to", not "Fixes"). The issue discussion also describes (1) a second crash: repeated results with the same UID that mix an explicit TrxTestDefinitionName with an omitted one whose display-name fallback differs, and (2) a proposal to report genuine contract violations as controlled reporting failures (preserve recoverable results, mark the run unsuccessful, record the affected IDs in RunInfo, nonzero exit code). This PR implements the first part of the stated preference ("support source-less tests through a documented fallback") only.

Design and decision record

This is a bug fix, not a new feature, so there is no separate design document; the rationale is here.

• Why a fallback instead of keeping the throw. TestMethod/@className is use="required" in the TRX schema (vstst.xsd, vendored copy; the XSD ships with Visual Studio and is not in this repo), so the report needs some class name, but a test node is not obliged to have one. The engine already accepts a missing method identifier whenever TrxFullyQualifiedTypeNameProperty is present (the method name falls back to the display name, which is how NUnit is reported today), so "neither property" was the only remaining hard failure. Failing the whole report, with exit code 7 and an empty TRX, for a node that has a valid final state is disproportionate.
• A comment I changed. The NOTE above that display-name fallback ended with "However, this is very wrong and we probably should fail if TestMethodIdentifierProperty isn't present." This PR deliberately does the opposite for source-less nodes (the method-name fallback itself is unchanged), so I replaced that sentence with the new decision instead of leaving a comment that contradicts the code. Easy to restore if you would rather keep the original remark.
• Why DefaultClassName. It is the last-resort class name of the VSTest TRX logger: Converter.GetTestClassName starts with var className = "DefaultClassName"; (microsoft/vstest@0f5ac692, Converter.cs#L593-L595). TRX consumers that already handle VSTest output see the same placeholder. There was no existing constant or convention in src/Platform to reuse.
• Alternatives considered (not implemented).
  • Module/assembly name as the class name: environment-dependent and adds edge cases (empty or odd names).
  • Node UID or display name as the class name: creates one TRX class per test and UID formats are framework-specific. Frameworks that want a specific grouping can still supply TrxFullyQualifiedTypeNameProperty, which keeps precedence (the issue author says they will do that framework-side).
  • An empty class name: the attribute is required and an empty value is not useful to TRX consumers.
  • Keeping the throw with a clearer message: still crashes the host for a valid source-less node.
• Known trade-offs. Multiple class-less nodes in one test app share the DefaultClassName class; each is still its own test definition with its own test id. An empty fully qualified type name, or a method identifier with an empty type name, can still produce an empty class name (pre-existing and unchanged).

Demonstration

Real composed product, no mocks: a minimal Microsoft Testing Platform host (source at the end of this section) whose test framework reports only a final state and timing, the shape from the issue, plus the real Microsoft.Testing.Extensions.TrxReport assemblies. The same host source was built twice in a scratch project outside the repo (nothing of it is committed): once against the released package, once against this branch's build output.

Before: released Microsoft.Testing.Extensions.TrxReport 2.5.3 from nuget.org (the shipped bits users have, not this PR's merge base):

$env:E2E_SHAPE = 'state-only'
.\E2EReleased.exe --report-trx --report-trx-filename e2e.trx --results-directory results-before

Unhandled exception. System.InvalidOperationException: The test framework 'StateOnlyTestFramework' with UID 'StateOnlyTestFramework' does not support the 'ITrxReportCapability' leading to missing or incomplete TRX reports
   at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.GetClassAndMethodName(TrxTestResult testResult)
   at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.AddResults(...)
   ...
Test host crashed and no test results could be recovered from the TRX streaming sidecar; the TRX is empty.
Test application process didn't exit gracefully, exit code is '-532462766'
host exit code: 7

(host exit code is $LASTEXITCODE of the launched executable, i.e. the parent test application; the child test host process is the one that crashed with -532462766.) The generated TRX has ResultSummary outcome="Failed", all counters 0, no UnitTest/UnitTestResult entries, and a single RunInfo error "Test host process pid: ... crashed.".

After: this branch's Debug build outputs (net10.0; content identical to a4eef71cb38972a0b0a9219652b02c5fd48bb640), same input:

Test run summary: Passed!
  total: 1
  failed: 0
  succeeded: 1
  skipped: 0
host exit code: 0

ResultSummary outcome=Completed total=1 passed=1 failed=0
TestMethod className=DefaultClassName name=bare display name
UnitTestResult testName=bare display name outcome=Passed duration=00:00:02.0000000

Nodes that do carry type information are unchanged. With E2E_SHAPE=all (default) the host publishes three nodes; the run exits 0 with 3 passed, ResultSummary Completed, and every UnitTestResult Passed with duration 00:00:02.0000000:

Provenance. "Before" used the assemblies from the 2.5.3 package (product version 2.5.3+0b1d8b5f7b2186835e2cf26292f3478f7cca087e), referenced with <PackageReference Include="Microsoft.Testing.Extensions.TrxReport" Version="2.5.3" />. "After" referenced this branch's Debug net10.0 outputs by HintPath (Microsoft.Testing.Platform, Microsoft.Testing.Extensions.TrxReport.Abstractions, Microsoft.Testing.Extensions.TrxReport); the SHA-256 of the scratch copies equals the build output:

using Microsoft.Testing.Extensions;
using Microsoft.Testing.Extensions.TrxReport.Abstractions;
using Microsoft.Testing.Platform.Builder;
using Microsoft.Testing.Platform.Capabilities.TestFramework;
using Microsoft.Testing.Platform.Extensions.Messages;
using Microsoft.Testing.Platform.Extensions.TestFramework;

ITestApplicationBuilder builder = await TestApplication.CreateBuilderAsync(args);
builder.RegisterTestFramework(
    _ => new TestFrameworkCapabilities(new TrxReportCapability()),
    (_, _) => new StateOnlyTestFramework());
builder.AddTrxReportProvider();
using ITestApplication app = await builder.BuildAsync();
return await app.RunAsync();

internal sealed class TrxReportCapability : ITrxReportCapability
{
    bool ITrxReportCapability.IsSupported => true;

    void ITrxReportCapability.Enable()
    {
    }
}

// Mimics a framework (for example xUnit) that only reports a final state and timing.
// E2E_SHAPE=state-only publishes just that node; E2E_SHAPE=all (default) also publishes nodes
// carrying the optional type information, to show that their class names are unchanged.
internal sealed class StateOnlyTestFramework : ITestFramework, IDataProducer
{
    public string Uid => nameof(StateOnlyTestFramework);

    public string Version => "1.0.0";

    public string DisplayName => nameof(StateOnlyTestFramework);

    public string Description => nameof(StateOnlyTestFramework);

    public Type[] DataTypesProduced => [typeof(TestNodeUpdateMessage)];

    public Task<bool> IsEnabledAsync() => Task.FromResult(true);

    public Task<CreateTestSessionResult> CreateTestSessionAsync(CreateTestSessionContext context)
        => Task.FromResult(new CreateTestSessionResult { IsSuccess = true });

    public Task<CloseTestSessionResult> CloseTestSessionAsync(CloseTestSessionContext context)
        => Task.FromResult(new CloseTestSessionResult { IsSuccess = true });

    public async Task ExecuteRequestAsync(ExecuteRequestContext context)
    {
        DateTimeOffset start = new(2026, 9, 27, 12, 0, 0, TimeSpan.Zero);
        var timing = new TimingProperty(new TimingInfo(start, start.AddSeconds(2), TimeSpan.FromSeconds(2)));
        bool all = (Environment.GetEnvironmentVariable("E2E_SHAPE") ?? "all") == "all";

        await PublishAsync(context, "bare", "bare display name", new PropertyBag(PassedTestNodeStateProperty.CachedInstance, timing));

        if (all)
        {
            await PublishAsync(
                context,
                "fqtn",
                "fqtn display name",
                new PropertyBag(PassedTestNodeStateProperty.CachedInstance, timing, new TrxFullyQualifiedTypeNameProperty("Fqn.Namespace.FqnType")));

            await PublishAsync(
                context,
                "ident",
                "ident display name",
                new PropertyBag(
                    PassedTestNodeStateProperty.CachedInstance,
                    timing,
                    new TestMethodIdentifierProperty("asm", "My.Namespace", "MyType", "MyMethod", 0, [], "System.Void")));
        }

        context.Complete();
    }

    private async Task PublishAsync(ExecuteRequestContext context, string uid, string displayName, PropertyBag properties)
        => await context.MessageBus.PublishAsync(
            this,
            new TestNodeUpdateMessage(context.Request.Session.SessionUid, new TestNode { Uid = uid, DisplayName = displayName, Properties = properties }));
}

Validation and regression proof

Environment: Windows, PowerShell. The repo-pinned SDK 12.0.100-alpha.1.26556.115 was installed by eng\build.ps1 into the git-ignored repo-local .dotnet folder (the machine's global SDK is 11.0.100). Source state: this branch at a4eef71cb38972a0b0a9219652b02c5fd48bb640. The binaries were built before committing, from a working tree whose content is identical to the commit (file hashes and timestamps checked; the worktree is clean at the commit).

Build (warnings are errors in this repo):

.\eng\build.ps1 -restore -build -projects "test\UnitTests\Microsoft.Testing.Extensions.UnitTests\Microsoft.Testing.Extensions.UnitTests.csproj" -configuration Debug -binaryLog -verbosity minimal

Result: Build succeeded. with 0 Warning(s) and 0 Error(s), exit code 0. This builds Microsoft.Testing.Extensions.TrxReport and Microsoft.Testing.Extensions.TrxReport.Abstractions (netstandard2.0, net10.0, net11.0) and the unit-test project (net462, net472, net10.0, net11.0). -installWindowsSdk was not used.

Tests (the test app is an MTP executable; run directly with the repo-local runtime):

$env:DOTNET_ROOT = "<repo>\.dotnet"; $env:DOTNET_MULTILEVEL_LOOKUP = "0"
$exe = "artifacts\bin\Microsoft.Testing.Extensions.UnitTests\Debug\<tfm>\Microsoft.Testing.Extensions.UnitTests.exe"
& $exe --filter "Name~OnlyStateAndTiming|Name~WithTestMethodIdentifier_ClassName" --progress off --output Normal
& $exe --filter "FullyQualifiedName~UnitTests.Trx|FullyQualifiedName~UnitTests.RetryTrxRegistrationBehaviorTests|FullyQualifiedName~TrxReport.Abstractions.UnitTests" --progress off --output Normal

The 11 skips on net10.0/net11.0 are pre-existing Linux-only link/merge tests (Merge_Linux*, MergeTool_Linux*, TemporarySibling_Linux*, TrxMerge_DanglingDestinationLink, TrxMerge_ExistingDestinationLink) reported as NotExecuted on Windows; none relate to this change.

Red/green. On the same tree, the two production files (TrxReportEngine.cs, TrxReportEngine.Results.cs) were restored to origin/main (e92578e3a4c076150c8b4174fbe04bd5b9df91d6) with the new tests kept, rebuilt (0 warnings, 0 errors) and the 4 new cases run on net10.0. Red:

failed TrxReportEngine_GenerateReportAsync_WithOnlyStateAndTiming_UsesFallbackClassNameAndDisplayNameAsMethodName
  System.InvalidOperationException: The test framework '' with UID '' does not support the 'ITrxReportCapability' leading to missing or incomplete TRX reports
    at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.GetClassAndMethodName(TrxTestResult testResult) in TrxReportEngine.Results.cs:284
    at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.AddResults(...) in TrxReportEngine.Results.cs:239
failed Session_NodeWithOnlyStateAndTimingStreamsAndReportsFallbackClassName
  System.InvalidOperationException: The test framework 'Fixture framework' with UID 'fixture.adapter' does not support the 'ITrxReportCapability' leading to missing or incomplete TRX reports
    at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.GetClassAndMethodName(TrxTestResult testResult) in TrxReportEngine.Results.cs:284
    at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.AddResults(...) in TrxReportEngine.Results.cs:239
Test run summary: Failed!
  total: 4
  failed: 2
  succeeded: 2
  skipped: 0

Green: with the fix restored and rebuilt, the same 4 cases give total: 4, failed: 0, succeeded: 4, skipped: 0 (table above).

Pitfall found while doing this: right after restoring the fixed files, MSBuild's incremental compile skipped them (the restored files kept old timestamps), so a first "green" run still executed the old code and reproduced the throw. I touched the files, rebuilt (a real 1m58s recompile), confirmed the assembly timestamp, and re-ran; every green number above comes from fresh binaries.

Static checks: git diff --check is clean; git diff --stat origin/main..HEAD is 6 files, 114 insertions, 7 deletions (3 C# files and one PACKAGE.md under src/Platform/, 2 test files); no PublicAPI.*.txt, .resx/.xlf, global.json or .lock.yml file is touched.

Test-value evidence

Tested revision: this branch; counterfactual: the same tree with the two production files restored to origin/main.

Red: total: 4, failed: 2, succeeded: 2 (the two state-only tests). Green: total: 4, failed: 0, succeeded: 4. Existing tests of the unchanged failure paths (duplicate singleton properties: TrxPropertyProjectionTests.Extract_DuplicateSingleton_RejectsAmbiguousReportMetadata and TrxTestResultExtractorTests.Extract_DuplicateSingletonProperty_Throws; non-final state: TrxTestResultExtractorTests.Extract_InProgressState_ThrowsUnreachableException) are untouched and pass in the "All TRX-related test classes" runs.

Not validated / evidence gaps: CI has not run (draft). Linux and macOS were not run. The acceptance/integration test projects were not run (they need packed packages); the demonstration above runs the real host against build output, not packed packages. Only the TRX-related classes of the unit-test project were run, not the whole project. The second crash and controlled-failure reporting described under "Out of scope" are not covered.