Add "exposed" operation for system nexus endpoint #736
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Member
There was a problem hiding this comment. This should go in nexus-rpc-gen eventually but it's okay if it starts out in this repo.
Member
There was a problem hiding this comment. I'm fine with what you have here because it gets the job done but the task was to add a proper tool that anybody can use, not something that is specific to our API repo.
Contributor
Author
There was a problem hiding this comment. I am tracking this work: nexus-rpc/nexus-rpc-gen#2, this is a prototype to make sure we have the generated code we want. It will be easily to copy this to the nexus-rpc/nexus-rpc-gen once we have that set up
Member
There was a problem hiding this comment. Feels like this file deserves the most scrutiny but if we end up not using the JSON schema based models it's not worth putting more effort into it. Where did you come up with this logic? Did you copy it from grpc-gateway? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,203 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "slices" | ||
| "sort" | ||
| "strings" | ||
|
|
||
| nexusannotationsv1 "github.com/nexus-rpc/nexus-proto-annotations/go/nexusannotations/v1" | ||
| "google.golang.org/protobuf/compiler/protogen" | ||
| "google.golang.org/protobuf/proto" | ||
| "google.golang.org/protobuf/reflect/protoreflect" | ||
| "google.golang.org/protobuf/types/descriptorpb" | ||
| "gopkg.in/yaml.v3" | ||
| ) | ||
|
|
||
| func generate(gen *protogen.Plugin) error { | ||
| nexusDoc := newDoc() | ||
| langsDoc := newDoc() | ||
|
|
||
| for _, f := range gen.Files { | ||
| if !f.Generate { | ||
| continue | ||
| } | ||
| for _, svc := range f.Services { | ||
| for _, m := range svc.Methods { | ||
| opts, ok := m.Desc.Options().(*descriptorpb.MethodOptions) | ||
| if !ok || opts == nil { | ||
| continue | ||
| } | ||
| if !proto.HasExtension(opts, nexusannotationsv1.E_Operation) { | ||
| continue | ||
| } | ||
| opOpts := proto.GetExtension(opts, nexusannotationsv1.E_Operation).(*nexusannotationsv1.OperationOptions) | ||
| if !slices.Contains(opOpts.GetTags(), "exposed") { | ||
|
Member
There was a problem hiding this comment. See what https://github.com/bergundy/protoc-gen-go-nexus did here, don't hardcode |
||
| continue | ||
| } | ||
|
|
||
| svcName := string(svc.Desc.Name()) | ||
| methodName := string(m.Desc.Name()) | ||
|
|
||
| addOperation(nexusDoc, svcName, methodName, | ||
| map[string]string{"$ref": openAPIRef(m.Input.Desc)}, | ||
| map[string]string{"$ref": openAPIRef(m.Output.Desc)}, | ||
| ) | ||
|
|
||
| addOperation(langsDoc, svcName, methodName, | ||
| langRefs(f.Desc, m.Input.Desc), | ||
| langRefs(f.Desc, m.Output.Desc), | ||
| ) | ||
| } | ||
| } | ||
| } | ||
| if err := writeFile(gen, "nexus/temporal-json-schema-models-nexusrpc.yaml", nexusDoc); err != nil { | ||
| return err | ||
| } | ||
| return writeFile(gen, "nexus/temporal-proto-models-nexusrpc.yaml", langsDoc) | ||
|
Member
There was a problem hiding this comment. These paths needs to be parameters, not hardcoded. |
||
| } | ||
|
|
||
| // openAPIRef returns the nexus-rpc-gen multi-file $ref string for a message type, | ||
| // referencing the openapiv3.yaml components/schemas entry. The path is relative to nexus/. | ||
| // | ||
| // Schema key convention used by protoc-gen-openapi (v3): | ||
| // | ||
| // {MessageName} (no package prefix) | ||
| // | ||
| // e.g. message "SignalWithStartWorkflowExecutionRequest" | ||
| // → "../openapi/openapiv3.yaml#/components/schemas/SignalWithStartWorkflowExecutionRequest" | ||
| func openAPIRef(msg protoreflect.MessageDescriptor) string { | ||
|
Member
There was a problem hiding this comment. When you write a proper protoc plugin that is reusable, you wouldn't be able to rely on openapi schemas being available in these paths.
Contributor
Author
There was a problem hiding this comment. I've duplicated this code into a draft PR in nexus-rpc/nexus-rpc-gen while we sort out that story: nexus-rpc/nexus-rpc-gen#36 |
||
| return "../openapi/openapiv3.yaml#/components/schemas/" + string(msg.Name()) | ||
| } | ||
|
|
||
| // langRefs builds the map of language-specific type refs for a message. | ||
| // Go, Java, dotnet, and Ruby refs are derived from proto file-level package options. | ||
| // Python and TypeScript refs are derived from the go_package path, taking only the | ||
| // last two path segments ({service}/v{n}). | ||
| func langRefs(file protoreflect.FileDescriptor, msg protoreflect.MessageDescriptor) map[string]string { | ||
| opts, ok := file.Options().(*descriptorpb.FileOptions) | ||
| if !ok || opts == nil { | ||
| return nil | ||
| } | ||
| name := string(msg.Name()) | ||
| refs := make(map[string]string) | ||
|
|
||
| if pkg := opts.GetGoPackage(); pkg != "" { | ||
| // strip the ";alias" suffix (e.g. "go.temporal.io/api/workflowservice/v1;workflowservice") | ||
| pkg = strings.SplitN(pkg, ";", 2)[0] | ||
| refs["$goRef"] = pkg + "." + name | ||
|
|
||
| segments := strings.Split(pkg, "/") | ||
| if len(segments) >= 2 { | ||
| tail := strings.Join(segments[len(segments)-2:], "/") | ||
| refs["$pythonRef"] = "temporalio.api." + strings.ReplaceAll(tail, "/", ".") + "." + name | ||
| refs["$typescriptRef"] = "@temporalio/api/" + tail + "." + name | ||
| } | ||
| } | ||
| if pkg := opts.GetJavaPackage(); pkg != "" { | ||
| refs["$javaRef"] = pkg + "." + name | ||
| } | ||
| if pkg := opts.GetRubyPackage(); pkg != "" { | ||
| refs["$rubyRef"] = pkg + "::" + name | ||
| } | ||
| if pkg := opts.GetCsharpNamespace(); pkg != "" { | ||
| refs["$dotnetRef"] = pkg + "." + name | ||
| } | ||
| if len(refs) == 0 { | ||
| return nil | ||
| } | ||
| return refs | ||
| } | ||
|
|
||
| // newDoc creates a yaml.Node document with the "nexusrpc: 1.0.0" header | ||
| // and an empty "services" mapping node, returned as a *yaml.Node (document node). | ||
| func newDoc() *yaml.Node { | ||
| doc := &yaml.Node{Kind: yaml.DocumentNode} | ||
| root := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"} | ||
| doc.Content = []*yaml.Node{root} | ||
|
|
||
| root.Content = append(root.Content, | ||
| scalarNode("nexusrpc"), | ||
| scalarNode("1.0.0"), | ||
| scalarNode("services"), | ||
| &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"}, | ||
| ) | ||
| return doc | ||
| } | ||
|
|
||
| // servicesNode returns the "services" mapping node from a doc created by newDoc. | ||
| func servicesNode(doc *yaml.Node) *yaml.Node { | ||
| root := doc.Content[0] | ||
| for i := 0; i < len(root.Content)-1; i += 2 { | ||
| if root.Content[i].Value == "services" { | ||
| return root.Content[i+1] | ||
| } | ||
| } | ||
| panic("services node not found") | ||
| } | ||
|
|
||
| // addOperation inserts a service → operation → {input, output} entry into doc. | ||
| // Services and operations are inserted in the order first encountered. | ||
| func addOperation(doc *yaml.Node, svcName, methodName string, input, output map[string]string) { | ||
| svcs := servicesNode(doc) | ||
|
|
||
| // find or create service node | ||
| var svcOps *yaml.Node | ||
| for i := 0; i < len(svcs.Content)-1; i += 2 { | ||
| if svcs.Content[i].Value == svcName { | ||
| // find "operations" within service mapping | ||
| svcMap := svcs.Content[i+1] | ||
| for j := 0; j < len(svcMap.Content)-1; j += 2 { | ||
| if svcMap.Content[j].Value == "operations" { | ||
| svcOps = svcMap.Content[j+1] | ||
| } | ||
| } | ||
| } | ||
| } | ||
| if svcOps == nil { | ||
| svcMap := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"} | ||
| svcOps = &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"} | ||
| svcMap.Content = append(svcMap.Content, | ||
| scalarNode("operations"), | ||
| svcOps, | ||
| ) | ||
| svcs.Content = append(svcs.Content, scalarNode(svcName), svcMap) | ||
| } | ||
|
|
||
| // build operation node | ||
| opNode := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"} | ||
| if len(input) > 0 { | ||
| opNode.Content = append(opNode.Content, scalarNode("input"), mapNode(input)) | ||
| } | ||
| if len(output) > 0 { | ||
| opNode.Content = append(opNode.Content, scalarNode("output"), mapNode(output)) | ||
| } | ||
| svcOps.Content = append(svcOps.Content, scalarNode(methodName), opNode) | ||
| } | ||
|
|
||
| // mapNode serializes a map[string]string as a yaml mapping node with keys in sorted order. | ||
| func mapNode(m map[string]string) *yaml.Node { | ||
| keys := make([]string, 0, len(m)) | ||
| for k := range m { | ||
| keys = append(keys, k) | ||
| } | ||
| sort.Strings(keys) | ||
| node := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"} | ||
| for _, k := range keys { | ||
| node.Content = append(node.Content, scalarNode(k), scalarNode(m[k])) | ||
| } | ||
| return node | ||
| } | ||
|
|
||
| func scalarNode(value string) *yaml.Node { | ||
| return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: value} | ||
| } | ||
|
|
||
| func writeFile(gen *protogen.Plugin, name string, doc *yaml.Node) error { | ||
| f := gen.NewGeneratedFile(name, "") | ||
| enc := yaml.NewEncoder(f) | ||
| enc.SetIndent(2) | ||
| if err := enc.Encode(doc); err != nil { | ||
| return err | ||
| } | ||
| return enc.Close() | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| module github.com/temporalio/api/cmd/protoc-gen-nexus-rpc-yaml | ||
|
|
||
| go 1.25.4 | ||
|
|
||
| require ( | ||
| google.golang.org/protobuf v1.36.1 | ||
| gopkg.in/yaml.v3 v3.0.1 | ||
| ) | ||
|
|
||
| require github.com/nexus-rpc/nexus-proto-annotations v0.0.0-20260330194009-e558d6edaf84 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU= | ||
| github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= | ||
| github.com/nexus-rpc/nexus-proto-annotations v0.0.0-20260330194009-e558d6edaf84 h1:SWHt3Coj0VvF0Km1A0wlY+IjnHKsjQLgO29io84r3wY= | ||
| github.com/nexus-rpc/nexus-proto-annotations v0.0.0-20260330194009-e558d6edaf84/go.mod h1:n3UjF1bPCW8llR8tHvbxJ+27yPWrhpo8w/Yg1IOuY0Y= | ||
| golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4= | ||
| golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= | ||
| google.golang.org/protobuf v1.36.1 h1:yBPeRvTftaleIgM3PZ/WBIZ7XM/eEYAaEyCwvyjq/gk= | ||
| google.golang.org/protobuf v1.36.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= | ||
| gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= | ||
| gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= | ||
| gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= | ||
| gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| // protoc-gen-nexus-rpc-yaml is a protoc plugin that generates nexus/nexusrpc.yaml | ||
| // and nexus/nexusrpc.langs.yaml from proto service methods annotated with | ||
| // option (nexusannotations.v1.operation).tags = "exposed". | ||
| package main | ||
|
|
||
| import ( | ||
| "google.golang.org/protobuf/compiler/protogen" | ||
| ) | ||
|
|
||
| func main() { | ||
| protogen.Options{}.Run(func(gen *protogen.Plugin) error { | ||
| return generate(gen) | ||
| }) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| nexusrpc: 1.0.0 | ||
| services: | ||
| WorkflowService: | ||
| operations: | ||
| SignalWithStartWorkflowExecution: | ||
| input: | ||
| $ref: ../openapi/openapiv3.yaml#/components/schemas/SignalWithStartWorkflowExecutionRequest | ||
| output: | ||
| $ref: ../openapi/openapiv3.yaml#/components/schemas/SignalWithStartWorkflowExecutionResponse |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| nexusrpc: 1.0.0 | ||
| services: | ||
| WorkflowService: | ||
| operations: | ||
| SignalWithStartWorkflowExecution: | ||
| input: | ||
| $dotnetRef: Temporalio.Api.WorkflowService.V1.SignalWithStartWorkflowExecutionRequest | ||
| $goRef: go.temporal.io/api/workflowservice/v1.SignalWithStartWorkflowExecutionRequest | ||
| $javaRef: io.temporal.api.workflowservice.v1.SignalWithStartWorkflowExecutionRequest | ||
| $pythonRef: temporalio.api.workflowservice.v1.SignalWithStartWorkflowExecutionRequest | ||
| $rubyRef: Temporalio::Api::WorkflowService::V1::SignalWithStartWorkflowExecutionRequest | ||
| $typescriptRef: '@temporalio/api/workflowservice/v1.SignalWithStartWorkflowExecutionRequest' | ||
| output: | ||
| $dotnetRef: Temporalio.Api.WorkflowService.V1.SignalWithStartWorkflowExecutionResponse | ||
| $goRef: go.temporal.io/api/workflowservice/v1.SignalWithStartWorkflowExecutionResponse | ||
| $javaRef: io.temporal.api.workflowservice.v1.SignalWithStartWorkflowExecutionResponse | ||
| $pythonRef: temporalio.api.workflowservice.v1.SignalWithStartWorkflowExecutionResponse | ||
| $rubyRef: Temporalio::Api::WorkflowService::V1::SignalWithStartWorkflowExecutionResponse | ||
| $typescriptRef: '@temporalio/api/workflowservice/v1.SignalWithStartWorkflowExecutionResponse' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package nexusannotations.v1; | ||
|
|
||
| import "google/protobuf/descriptor.proto"; | ||
|
|
||
| option go_package = "github.com/nexus-rpc/nexus-proto-annotations/go/nexusannotations/v1"; | ||
|
|
||
| extend google.protobuf.ServiceOptions { | ||
| optional ServiceOptions service = 8233; | ||
| } | ||
|
|
||
| extend google.protobuf.MethodOptions { | ||
| optional OperationOptions operation = 8234; | ||
| } | ||
|
|
||
| message OperationOptions { | ||
| // Nexus operation name (defaults to proto method name). | ||
| string name = 1; | ||
| // Tags to attach to the operation. Used by code generators to include and exclude operations. | ||
| repeated string tags = 2; | ||
| } | ||
|
|
||
| message ServiceOptions { | ||
|
Member
There was a problem hiding this comment. We have tags on services too but you didn't support excluding or including based on tags. That's fine for now, but the "official" plugin would need to have all of this functionality, similar to what |
||
| // Nexus service name (defaults to proto service full name). | ||
| string name = 1; | ||
| // Tags to attach to the service. Used by code generators to include and exclude services. | ||
| repeated string tags = 2; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why don't we have the same make target for google? Seems like we can sync both. But probably don't sync google as part of this PR if there are any changes there, better to do that separately.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a lot of changes in the google/api definitions that we don't have in this repo, do we want to add all of those all at once?
https://buf.build/googleapis/googleapis/docs/main%3Agoogle.api
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, separate PR please.