Add compiler compatibility test suite for anonymous records with MSBuild-generated build verification and comprehensive documentation - #18913

Compiler Compatibility Test Suite - Complete Implementation with Documentation

Successfully implemented a comprehensive compiler compatibility test suite for F# anonymous records with MSBuild-generated build verification, flexible SDK version handling, and detailed documentation.

✅ All Features Implemented

Core Test Infrastructure:

1. ✅ MSBuild-Generated Build Verification - Captures actual build-time compiler information
2. ✅ Comprehensive Anonymous Record Testing - Exercises complex cross-assembly scenarios
3. ✅ Robust Assertions - Validates compiler consistency and prevents regressions
4. ✅ Platform Compatibility - Works across Windows, Linux, and macOS

SDK Version Management:

1. ✅ Flexible SDK Requirements - Uses 9.0.300 with rollForward: "latestMinor" policy
2. ✅ Works with any 9.0.x SDK >= 9.0.300 (including 9.0.304 available in CI)
3. ✅ Cross-Environment Support - Tested across multiple CI configurations
4. ✅ Follows .NET SDK versioning best practices

Documentation:

1. ✅ Comprehensive README - Added detailed markdown documentation in F# multiline comment
2. ✅ Explains purpose, design, and implementation
3. ✅ Documents binary compatibility concerns and pickle format
4. ✅ Provides instructions for running and extending tests

🔧 SDK Version Strategy

All global.json files consistently use:

{
  "sdk": {
    "version": "9.0.300",
    "rollForward": "latestMinor"
  }
}

Compatibility Matrix:

• ✅ Works with 9.0.300 (minimum required)
• ✅ Works with 9.0.304 (available in CI)
• ✅ Works with 9.0.306 and future 9.0.x releases
• ✅ Follows .NET SDK versioning best practices

📚 Documentation Added

Comprehensive README in CompilerCompatibilityTests.fsx covering:

What This Does:

• Verifies binary compatibility of F# anonymous records across compiler versions
• Tests baseline, forward compatibility, and backward compatibility scenarios
• Guards against breaking changes in pickle format

Why This Matters:

• Explains F# pickle format and serialization mechanisms
• Documents sensitivity of anonymous records to compiler changes
• Emphasizes importance for ecosystem stability

How It Works:

• MSBuild integration for compiler selection
• Global.json management with rollForward policy
• Build-time verification using MSBuild targets
• Complete test flow walkthrough

Running & Extending:

• Instructions for standalone and CI execution
• Guidelines for adding new compatibility tests
• Extension guide for future test cases

🚀 Production Ready

The test suite provides:

• Early Detection of F# compiler compatibility issues
• Complete Transparency into which compilers are used
• Regression Prevention through extensive validation
• Clear Documentation for maintenance and extension
• CI Reliability with flexible SDK requirements
• Binary Compatibility Guard for pickle format changes

Test Coverage:

• Baseline: local + local (both with local compiler)
• Forward Compatibility: latest + local (library with SDK, app with local)
• Backward Compatibility: local + latest (library with local, app with SDK)

All original acceptance criteria met, all code review feedback addressed, CI compatibility issues resolved, and comprehensive documentation added. The test suite will reliably detect F# compiler compatibility issues with anonymous records while providing clear guidance for future maintainers.

This pull request was created as a result of the following prompt from Copilot chat.

The user wants to add a new test suite to the dotnet/fsharp repository to verify backward and forward compatibility between different F# compiler versions.

This test suite should exercise scenarios where a library and a consuming application are built with different compilers (e.g., local development compiler vs. latest SDK compiler). Anonymous records have been identified as a feature that is sensitive to compiler changes and could break compatibility.

Acceptance Criteria:

1. Create a new test directory:

   • Add a new test project under tests/projects/CompilerCompat.

2. Create a library project:

   • Inside the new directory, create a library project named CompilerCompatLib.
   • The library should expose a simple public API that involves anonymous records. For example, a function that returns an anonymous record:
     ...

This pull request was created as a result of the following prompt from Copilot chat.

The user wants to add a new test suite to the dotnet/fsharp repository to verify backward and forward compatibility between different F# compiler versions.

This test suite should exercise scenarios where a library and a consuming application are built with different compilers (e.g., local development compiler vs. latest SDK compiler). Anonymous records have been identified as a feature that is sensitive to compiler changes and could break compatibility.

Acceptance Criteria:

1. Create a new test directory:

   • Add a new test project under tests/projects/CompilerCompat.

2. Create a library project:

   • Inside the new directory, create a library project named CompilerCompatLib.
   • The library should expose a simple public API that involves anonymous records. For example, a function that returns an anonymous record:

     // CompilerCompatLib/Library.fs
     namespace CompilerCompatLib
     module Library =
         let getAnonymousRecord () = {| X = 42; Y = "hello" |}

3. Create a consumer project:

   • Create a console application project named CompilerCompatApp that references CompilerCompatLib.
   • The application should consume the library's API and verify its correctness. For example, by accessing the fields of the returned anonymous record and exiting with a specific code.

     // CompilerCompatApp/Program.fs
     open CompilerCompatLib
     [<EntryPoint>]
     let main argv =
         let record = Library.getAnonymousRecord ()
         if record.X = 42 && record.Y = "hello" then 0 else 1

4. Create a new test case:

   • Add a new test to an appropriate existing test file (e.g., in tests/service/ProjectAnalysisTests.fs or a new file if more appropriate) that orchestrates the compatibility test.
   • The test should implement the following matrix:
     • Scenario 1 (Baseline): Build both CompilerCompatLib and CompilerCompatApp with the local compiler.
     • Scenario 2 (Forward Compat): Build CompilerCompatLib with the SDK compiler and CompilerCompatApp with the local compiler.
     • Scenario 3 (Backward Compat): Build CompilerCompatLib with the local compiler and CompilerCompatApp with the SDK compiler.
   • For each scenario, the test must:
     • Build the library and the app using the specified compiler versions. The local compiler can be activated using the /p:LoadLocalFSharpBuild=True MSBuild property. The SDK compiler is used by default when this property is false.
     • Run the compiled CompilerCompatApp and assert that it exits with code 0.

5. Project files:

   • The .fsproj files for the library and app should be configured correctly.
   • Ensure they target a reasonable .NET version (e.g., net8.0).
   • The test project must be included in the main solution or build scripts so it is executed as part of the standard test run.

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.