Visitar URL original
Schema references in descriptions by Frassle · Pull Request #21369 · pulumi/pulumi · GitHub
Skip to content

Schema references in descriptions - #21369

Merged
Frassle merged 83 commits into
masterfrom
fraser/docrefs
Jun 15, 2026
Merged

Frassle merged 83 commits into
masterfrom
fraser/docrefs

Conversation

@Frassle

@Frassle Frassle commented Jan 5, 2026 •

Copy link
Copy Markdown
Member

Description fields can make reference to property names or other resources. These can be named differently in different language's SDKs. This is currently handled in the bridge by generating the schema once for each SDK language, but can't be rendered accurately in the registry.

We'll introduce a semantic reference tag to allow linking to elements within the local schema which can be specifically interpreted by each language and docs generation process.

The format is:

{{$ ref ... $}}

where the supported ref values are in the form:

#/resources/{token}
#/functions/{token}
#/types/{token}
#/resources/{token}/properties/{property}
#/resources/{token}/inputProperties/{property}
#/functions/{token}/inputs/properties/{property}
#/functions/{token}/outputs/properties/{property}
#/types/{token}/properties/{property}

Fixes #16099

Add a utility function to parse {{$ ref ... $}} tags from description fields. Parse the ref itself into a struct which the specific codegen implementations can easy work from.

danielrbradley and others added 26 commits March 31, 2025 11:31
Add a utility function to parse `<pulumi ref="" />` tags from description fields. Parse the ref itself into a struct which the specific codegen implementations can easy work from.

Implement with goldmark + html token parser for robustness.

- Include additional docs renderer to match behaviour of the examples filtering rendering, with extra override for HTML.
- Add DocRef constructor & IsWithin helper
- Avoid reformatting if no refs found
- Trim trailing newlines if they weren't present before
Render refs in all Python schema comments

- Use builtin methods for generating names.
- Leverage python name overrides for modules.
- Add "genComment" and make a member of *modContext so we can resolve module names.
@guineveresaenger

Copy link
Copy Markdown
Contributor

Couple of pre-review comments:

The bridge no longer generates a schema for each language as of pulumi/pulumi-terraform-bridge#3188. It's done in the bridge via a mark-up <span> tag on the Description field, so a more core-native way to markup would be great and fairly easy to pivot to.

Could you explain what you mean by:

[...] but can't be rendered accurately in the registry.

Maybe you're talking about fixUpPropertyReference in the bridge which does a best-guess approach for a lot of these references. The "best guess" part, though, is actually discovering these snippets - correctly verifying that they're pieces of code, or type/field references. The by-language translation works fairly well. Regardless, it would be amazing not to have to worry about this in the bridge but let SDKgen handle it. 💟

@Frassle

Frassle commented Jan 13, 2026

Copy link
Copy Markdown
Member Author

Maybe you're talking about fixUpPropertyReference in the bridge which does a best-guess approach for a lot of these references.

Yeh the intention is that tfbridge/registry shouldn't need to care about language specific bits in schemas at all. You write the schema once using {{% ref #/... %}} and then core codegen will deal with writing out the right per-language text.

Re-reading the tests for fixUpPropertyReference I noticed its also dealing with values like bools (true or True depending on language), so I might see if we can fit that into this scheme as well somehow.

Comment thread pkg/codegen/python/gen.go Outdated
case schema.DocRefKindResource, schema.DocRefKindResourceProperty:
base = mod.tokenToResource(ref.ResourceToken())
case schema.DocRefKindResourceInputProperty:
base = mod.tokenToResource(ref.ResourceToken()) + "Args"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometimes we call this InitArgs. This happens when there's a type with the same name as a resource.

resourceArgsName := name + "Args"
// Some providers (e.g. Kubernetes) have types with the same name as resources (e.g. StorageClass in Kubernetes).
// We've already shipped the input type (e.g. StorageClassArgs) in the same module as the resource, so we can't use
// the same name for the resource's args class. When an input type exists that would conflict with the name of the
// resource args class, we'll use a different name: `<Resource>InitArgs` instead of `<Resource>Args`.
const alternateSuffix = "InitArgs"
for _, t := range mod.types {
if mod.details(t).inputType {
if mod.unqualifiedObjectTypeName(t, true) == resourceArgsName {
resourceArgsName = name + alternateSuffix
break
}
}
}

Comment thread pkg/codegen/python/gen.go Outdated
case schema.DocRefKindFunctionOutputProperty:
base = title(PyName(tokenToName(ref.Function.Token))) + "Result"
case schema.DocRefKindType, schema.DocRefKindTypeProperty:
base = title(PyName(ref.Type.String()))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in Python schema.DocRefKindType are suffixed with Args

if t.IsInputShape() {
return name + "Args"
}
return name

@pgavlin pgavlin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks structurally correct! bunch of requests for doc comments + one note on cross-schema ref support.

Comment thread pkg/codegen/schema/docs.go Resolved
Comment thread pkg/codegen/schema/docs.go Outdated
Comment on lines +41 to +42
switch c := c.(type) {
case *Ref:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we're only handling *Ref--suggest a continue here:

Suggested change
switch c := c.(type) {
case *Ref:
cref, ok := c.(*Ref)
if !ok {
continue
}

Comment thread pkg/codegen/schema/docs.go Outdated
Comment on lines +127 to +144
var name string
var ok bool
if !subdiags.HasErrors() {
name, ok = resolveRefToName(ref)
}
diags = append(diags, subdiags...)

if !ok {
if ref.Property != "" {
name = ref.Property
} else if ref.Type != nil {
name = ref.Type.String()
} else if ref.Function != nil {
name = ref.Function.Token
} else {
name = ref.Ref
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment in here would be good as well

Comment thread pkg/codegen/schema/docs.go Resolved
Comment thread pkg/codegen/schema/docs.go Outdated
Comment on lines +272 to +284
// Parses a doc reference string into a DocRef struct.
// The supported formats of the ref is:
//
// #/resources/{token}
// #/functions/{token}
// #/types/{token}
// #/resources/{token}/properties/{property}
// #/resources/{token}/inputProperties/{property}
// #/functions/{token}/inputProperties/{property}
// #/functions/{token}/outputProperties/{property}
// #/types/{token}/properties/{property}
//
// Note: Tokens containing a slash ("/") must be encoded as "%2F".

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about referring to resources, etc. in other schemas? I think this needs to support a pre-fragment URI as well

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(can probably just say that path components inside the ref must be URL-encoded too, instead of referring only to /)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a last note: seems wise to include the parse error, if any, so that we can report malformed references

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be the same as type refs, but I'll add some tests for that.

Comment thread pkg/codegen/schema/docs_parser.go Resolved
Comment thread pkg/codegen/schema/docs.go Outdated
Comment on lines +34 to +35
// interpretPulumiRefs parses a {{% ref %}} shortcode node and passes it to the `resolveRefToName` function to replace
// the shortcode with literal text.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// interpretPulumiRefs parses a {{% ref %}} shortcode node and passes it to the `resolveRefToName` function to replace
// the shortcode with literal text.
// interpretPulumiRefs parses all {{% ref %}} shortcodes that descend from the given node. Each ref is passed to the `resolveRefToName` callback to replace
// the shortcode with literal text.

Comment on lines +290 to +304
// DocRef is a parsed and (when possible) bound reference to a schema entity that appears in a documentation
// string. It carries enough information for language-specific codegen to render the reference as a name in
// the target language.
type DocRef struct {
// Ref is the original ref string as it appeared in the source documentation (e.g. `#/resources/foo:bar:Baz`).
Ref string
// Kind identifies what sort of entity the ref points to. See the DocRefKind constants.
Kind DocRefKind
// Type is the bound schema type, if Kind refers to a resource or named type. Nil otherwise.
Type Type
// Function is the bound schema function, if Kind refers to a function or one of its properties. Nil otherwise.
Function *Function
// Property is the referenced property name for property-kind refs, or empty if the ref is to a top-level entity.
Property string
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to have a String function on this. could just return Ref, but could also return a normalized, package-qualified repr

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we either have to return Ref or always return a fully qualified path because given a Type you can't tell if it came from the "root" package or not.

return "docs"
}

func (p *DocsProvider) GetSchema(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this include a test for external refs?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeh probably, the logic is mostly binder side but doesn't hurt to check here.

@Frassle
Frassle added this pull request to the merge queue Jun 15, 2026
Merged via the queue into master with commit 9b28f98 Jun 15, 2026
230 of 234 checks passed
@Frassle
Frassle deleted the fraser/docrefs branch June 15, 2026 16:14
@pulumi-bot

Copy link
Copy Markdown
Contributor

This PR has been shipped in release v3.247.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Accurate property and resource references in descriptions

7 participants