Sample Generation - ExampleValueExpressionBuilder and OperationSample parity with autorest generator by MaiLinhP · Pull Request #10739 · microsoft/typespec

MaiLinhP · 2026-05-19T21:34:17Z

Implements M1-M4 milestones for sample generation for the C# emitter — from mock value synthesis to emitting compilable Samples_{Client}.cs files. Based on the design document.

New Components
ExampleMockValueBuilder (M1) — Generates InputOperationExample instances for operations without spec-provided examples. Produces ShortVersion (required only) and AllParameters (with optional parameters) variants. This was simplified compared to autorest versions - instead of giving concrete mock values that matches the expected format for time/duration/different number types etc., mock values were simplified to "<{parameter name/encoded type hints}>" for strings, and 0 for numbers. This is because we're expecting AI based sample generator to read from deterministic samples and come up with scenario based mock values, hence a hint would be more useful than hardcoded mock values.

ExampleValueExpressionBuilder (M2) — Converts InputExampleValue → C# ValueExpression AST nodes. Covers all primitives, collections, enums, models, credentials ApiKeyCredential, DefaultAzureCredential, BinaryContent anonymous objects, streams, etc.

ExampleParameterValue (M2) — Dual-mode bridge holding either raw InputExampleValue or a pre-built ValueExpression for known parameters (credentials, endpoints).

OperationSample (M3) — Resolves client construction chains (subclient → root), parameter value mappings (with spread support), inline/out-of-line decisions, and paging response type unwrapping.

ClientSampleProvider (M4) — TypeProvider that emits tests/Generated/Samples/Samples_{Client}.cs with sync+async [Test]/[Ignore] methods. Handles normal, pageable (foreach/await foreach), and LRO invocation patterns with JSON response parsing. Note: Per design doc, BuildSamples
should live in ScmMethodProviderCollection.cs which is the equivalent of autorest OperationMethodChainBuilder.cs. However, coupling sample building with method building was purely for convenience, so it has been moved out to ClientSampleProvider for more clarity.

Infrastructure Changes
ScmOutputLibrary — Registers ClientSampleProvider for all clients
ScmMethodProviderCollection — Exposed MethodProviders for sample provider access

pkg-pr-new · 2026-05-19T21:45:12Z

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-csharp@10739

commit: 7f31520

github-actions · 2026-05-19T21:55:51Z

No changes needing a change description found.

jorgerangel-msft

Sorry if I missed this, but do you have a design doc of the proposed sample generation pipeline? I think that will help with understanding where these changes fit in the overall picture.

jorgerangel-msft · 2026-05-26T17:54:10Z

            ];
        }

+        protected virtual IReadOnlyList<OperationSample> BuildSamples()


Is there a reason why this is protected virtual? Is it for testing? Also do you think the sample building should be decoupled from the client method construction we do here?

JoshLove-msft · 2026-05-27T22:11:41Z

Can we link to the design?

JoshLove-msft · 2026-05-27T22:45:47Z

Can we add some example data to our test TSP?

MaiLinhP · 2026-06-03T20:31:15Z

@JoshLove-msft design linked in description, and I added some examples while keeping some autogenerated mock examples for comparison :) Also interesting that examples are added using JSON load format, and TCGC doesn't seem to process the @opExample decorator at all? No .tsp file currently uses this decorator either 🤔

MaiLinhP · 2026-06-04T17:53:00Z

@copilot resolve the merge conflicts in this pull request

…autorest

…or sample generation on sample client

JoshLove-msft · 2026-06-05T00:46:16Z

Can we please get a preview pipeline run?

JoshLove-msft

Automated PR review — C# emitter sample generation (M1–M4)

Reviewed the non-generated source (ExampleMockValueBuilder, ExampleValueExpressionBuilder, ExampleParameterValue, OperationSample, ClientSampleProvider, and the supporting infra changes) plus spot-checked the generated Samples_*.cs output. Overall this is well-structured, thoroughly documented, and the generated bodies in the Local Sample-TypeSpec project look compilable. Test coverage is solid (~170 test methods across the new sample test files).

A few things worth addressing, left as inline comments:

Silent catch { continue; } in ClientSampleProvider.BuildMethods hides failures and drops samples with no diagnostic.
Reflection into the non-public CSharpType constructor (CreateExternalCSharpType) is brittle and unguarded by tests.
Generation-time numeric Convert calls on raw example strings in ExampleValueExpressionBuilder can throw FormatException for non-numeric spec example values.
No backtracking on visitedModels in ExampleMockValueBuilder.BuildModelExampleValue under-populates sibling properties that share a model type.
Unused using System.ClientModel; added to ScmKnownParameters.cs.

Minor / non-blocking: ConsoleWriteLine splices a member-access string as a method name; several known-parameter checks rely on type.Name == "WaitUntil" / "RequestConditions" string matching (acceptable given the cross-assembly setup); convenience samples declare an unused response local (compiles cleanly).

Note: the InputPrimitiveType.IsNumber expansion is low-risk — its only consumer in the generator is the new mock builder.

--generated by Copilot

JoshLove-msft · 2026-06-05T02:10:12Z

+                }
+                catch
+                {
+                    continue;


The bare catch { continue; } swallows every exception (including real bugs like NullReferenceException or contract violations) and silently drops the operation from the generated samples. If GetMethodCollectionByOperation starts throwing for a legitimate reason, samples will quietly go missing with no diagnostic. Consider catching a specific expected exception type, or at minimum emitting a diagnostic/warning through the generator's reporting so the drop is observable.

--generated by Copilot

JoshLove-msft · 2026-06-05T02:10:12Z

+                 typeof(CSharpType), typeof(IReadOnlyList<CSharpType>),
+                 typeof(bool), typeof(bool), typeof(CSharpType), typeof(Type)],
+                null)!;
+            return (CSharpType)ctor.Invoke([name, ns, false, false, null, Array.Empty<CSharpType>(), true, false, null, null]);


Invoking the non-public CSharpType constructor via reflection is brittle: it isn't checked by the compiler, so any change to that constructor's parameter list compiles fine here and then throws MissingMethodException/NullReferenceException at generation time. Is there a public/internal factory for external types (an existing CSharpType overload, or InternalsVisibleTo so the constructor can be called directly)? If reflection has to stay, please add a focused unit test that constructs _testAttributeType so the reflection contract is guarded against silent breakage.

--generated by Copilot

JoshLove-msft · 2026-06-05T02:10:12Z

+        /// Since the generated file already has <c>using System;</c>, this helper emits the shorter form.
+        /// </summary>
+        private static MethodBodyStatement ConsoleWriteLine(ValueExpression expression)
+            => new InvokeMethodExpression(null, "Console.WriteLine", [expression]).Terminate();


Passing the string Console.WriteLine as the method name to InvokeMethodExpression splices a member-access expression where a method identifier is expected. It works today only because the writer emits the string verbatim and using System; is present. This is fragile (it bypasses any identifier validation/escaping). Consider Static(typeof(Console)).Invoke(...) and, if the global::System.Console qualification is the concern, addressing that at the writer level rather than string-splicing a member access.

--generated by Copilot

JoshLove-msft · 2026-06-05T02:10:12Z

+            {
+                if (format == SerializationFormat.DateTime_Unix)
+                {
+                    var unixValue = rawValue is string us ? Convert.ToInt64(us) : rawValue is int or long ? Convert.ToInt64(rawValue) : 0L;


Convert.ToInt64(us) (and the analogous Convert.ToDouble(ds) for TimeSpan below) runs at generation time on the raw example string. For a spec-provided example whose unix-encoded DateTimeOffset / seconds-encoded TimeSpan value is a non-numeric placeholder string, this throws FormatException and crashes the whole emit. The synthesized mock builder currently emits numeric 0 here so it's safe today, but spec examples are not guaranteed to be numeric. Suggest long.TryParse / double.TryParse with a fallback to Default on failure.

--generated by Copilot

JoshLove-msft · 2026-06-05T02:10:12Z

+
+            var properties = new Dictionary<string, InputExampleValue>();
+            var result = InputExampleValue.Object(model, properties);
+            visitedModels.Add(model);


visitedModels.Add(model) is never removed, so this is cycle prevention without backtracking: if two sibling properties at the same level share a model type, the second one resolves to InputExampleValue.Null and the AllParameters mock ends up under-populated. Compare ClientSampleProvider.BuildResponseParseStatements, which does Add then Remove so the same type can appear on parallel branches. Recommend matching that backtracking pattern (remove the model from the set before returning) so only true recursion cycles are cut.

--generated by Copilot

JoshLove-msft · 2026-06-05T02:10:12Z

 // Licensed under the MIT License.

 using System;
+using System.ClientModel;


This using System.ClientModel; looks unused — nothing in the file references a type from that namespace root (ApiKeyCredential / ClientResult / BinaryContent are not used here). Appears to be a leftover from an earlier iteration; please remove it to avoid an unused-using warning.

--generated by Copilot

jorgerangel-msft · 2026-06-05T15:21:08Z

 [assembly: InternalsVisibleTo("Microsoft.TypeSpec.Generator.Input.Tests.Perf, PublicKey=002400000480000094000000060200000024000052534131000400000100010041df4fe80c5af6ff9a410db5a173b0ce24ad68764c623e308b1584a88b1d1d82277f746c1cccba48997e13db3366d5ed676576ffd293293baf42c643f008ba2e8a556e25e529c0407a38506555340749559f5100e6fd78cc935bb6c82d2af303beb0d3c6563400659610759b4ed5cb2e0faf36b17e6842f04cdc544c74e051ba")]
 [assembly: InternalsVisibleTo("Microsoft.TypeSpec.Generator.Input.Tests, PublicKey=002400000480000094000000060200000024000052534131000400000100010041df4fe80c5af6ff9a410db5a173b0ce24ad68764c623e308b1584a88b1d1d82277f746c1cccba48997e13db3366d5ed676576ffd293293baf42c643f008ba2e8a556e25e529c0407a38506555340749559f5100e6fd78cc935bb6c82d2af303beb0d3c6563400659610759b4ed5cb2e0faf36b17e6842f04cdc544c74e051ba")]
 [assembly: InternalsVisibleTo("Microsoft.TypeSpec.Generator.Tests.Common, PublicKey=002400000480000094000000060200000024000052534131000400000100010041df4fe80c5af6ff9a410db5a173b0ce24ad68764c623e308b1584a88b1d1d82277f746c1cccba48997e13db3366d5ed676576ffd293293baf42c643f008ba2e8a556e25e529c0407a38506555340749559f5100e6fd78cc935bb6c82d2af303beb0d3c6563400659610759b4ed5cb2e0faf36b17e6842f04cdc544c74e051ba")]
+[assembly: InternalsVisibleTo("Microsoft.TypeSpec.Generator, PublicKey=002400000480000094000000060200000024000052534131000400000100010041df4fe80c5af6ff9a410db5a173b0ce24ad68764c623e308b1584a88b1d1d82277f746c1cccba48997e13db3366d5ed676576ffd293293baf42c643f008ba2e8a556e25e529c0407a38506555340749559f5100e6fd78cc935bb6c82d2af303beb0d3c6563400659610759b4ed5cb2e0faf36b17e6842f04cdc544c74e051ba")]


Why is this needed? Typically we shouldn't expose the internals like this outside of tests. Should we consider making whatever types are needed non-internal instead ?

jorgerangel-msft · 2026-06-05T15:38:04Z

+            var methods = new List<MethodProvider>();
+
+            // Ensure client methods are built so MethodCache is populated
+            _ = _client.Methods;


should we be using the canonical view _client.CanonicalView.Methods ?

jorgerangel-msft · 2026-06-05T15:42:03Z

+            }
+            else
+            {
+                streamExpr = responseVar.Invoke("GetRawResponse", []).Property("ContentStream");


will this method invocation work for azure clients? In general, I recommend we use the ClientResponseApi whenever possible.

jorgerangel-msft · 2026-06-05T15:42:30Z

+                streamExpr = responseVar.Invoke("GetRawResponse", []).Property("ContentStream");
+            }
+
+            var parseExpr = Static(typeof(JsonDocument)).Invoke(nameof(JsonDocument.Parse), streamExpr)


nit: can we use the JsonDocumentSnippets?

jorgerangel-msft · 2026-06-05T15:43:49Z

+                        yield return stmt;
+                    }
+
+                    visitedTypes.Remove(modelType);


question: any chance of a race condition here between the add and remove ?

jorgerangel-msft · 2026-06-05T16:24:44Z

+                InputExampleValue.Stream(InputPrimitiveType.String, "<filePath>"));
+
+            // Should produce File.OpenRead("...")
+            Assert.IsInstanceOf<InvokeMethodExpression>(result);


This just validates that the built expression is of a certain type. Should we consider using TestData or validating the written expressions to ensure they are generated as we expect?

jorgerangel-msft · 2026-06-05T16:25:06Z

+            // (Format is private, but this is the key parity assertion)
+            var formatProp = typeof(FormattableStringExpression).GetProperty("Format", BindingFlags.NonPublic | BindingFlags.Instance);
+            Assert.IsNotNull(formatProp, "Format property should exist on FormattableStringExpression");
+            Assert.AreEqual("new DefaultAzureCredential()", formatProp!.GetValue(expr));


I don't think we should have azure related tests in MTG

jorgerangel-msft · 2026-06-05T16:28:30Z

+    {
+        /// <summary> This sample shows how to call SayHi and parse the result. </summary>
+        [global::NUnit.Framework.TestAttribute]
+        [global::NUnit.Framework.IgnoreAttribute("Only validating compilation of examples")]


Hmm do we know why these attributes weren't resolved by Roslyn ? We might have to add NUnit to the metadata references for the generator.

jorgerangel-msft · 2026-06-05T16:30:25Z

+            ApiKeyCredential credential = new ApiKeyCredential("<key>");
+            SampleTypeSpecClient client = new SampleTypeSpecClient(endpoint, credential);
+
+            using BinaryContent content = BinaryContent.Create(BinaryData.FromObjectAsJson(new


I think FromObjectAsJson is not AOT safe and may cause issues in the azure repo. I'm not sure if the tests directory is setup to catch this in that repo. @JoshLove-msft @jsquire do you know ?

jorgerangel-msft · 2026-06-05T16:31:34Z

+            {
+                requiredString = "<requiredString>",
+                requiredInt = 0,
+                requiredCollection = new object[] { "1" },


do you think we should try to use the explicit element type of the collection instead? will that work here?

microsoft-github-policy-service Bot added the emitter:client:csharp Issue for the C# client emitter: @typespec/http-client-csharp label May 19, 2026

MaiLinhP marked this pull request as ready for review May 21, 2026 17:29

MaiLinhP requested review from JoshLove-msft, ShivangiReja, jorgerangel-msft, joseharriaga, jsquire and m-nash as code owners May 21, 2026 17:29

jorgerangel-msft reviewed May 26, 2026

View reviewed changes

MaiLinhP force-pushed the sample branch from a15c414 to ed3739c Compare June 3, 2026 22:23

MaiLinhP added 11 commits June 4, 2026 12:17

OperationSample

ca8a3fd

Change TokenCredential to add DefaultAzureCredential for parity with …

37a75fe

…autorest

fix spelling

18188f3

Change to ScmOutputLibrary and ScmMethodProviderCollection to allow f…

002eabc

…or sample generation on sample client

minor form change

439f45b

Change credential type reference

f1083c8

Move Sample building into ClientSampleProvider

d720759

Run Generate.ps1

8076e7b

Add custom example samples to compare with auto generated mock versions

0b76293

regen change

d8f9f89

fix conflict

bce587f

MaiLinhP force-pushed the sample branch from ed3739c to bce587f Compare June 4, 2026 19:37

Fix tests to match expected behavior

7f31520

JoshLove-msft reviewed Jun 5, 2026

View reviewed changes

jorgerangel-msft reviewed Jun 5, 2026

View reviewed changes

Conversation

MaiLinhP commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 19, 2026

Uh oh!

jorgerangel-msft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JoshLove-msft commented May 27, 2026

Uh oh!

JoshLove-msft commented May 27, 2026

Uh oh!

MaiLinhP commented Jun 3, 2026

Uh oh!

MaiLinhP commented Jun 4, 2026

Uh oh!

JoshLove-msft commented Jun 5, 2026

Uh oh!

JoshLove-msft left a comment

Choose a reason for hiding this comment

Automated PR review — C# emitter sample generation (M1–M4)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MaiLinhP commented May 19, 2026 •

edited

Loading

pkg-pr-new Bot commented May 19, 2026 •

edited

Loading