[Go] Prompts with "WithTools" and "WithOutputType" results in error

[check24-axel-nitzsche]

Describe the bug
If we define a prompt with both ai.WithOutputType and ai.WithTool, the generated "toolRequest" result fail with the error:
model failed to generate output matching expected schema: data is not valid JSON: unexpected end of JSON input.

Without ai.WithOutputType, the "toolRequest" is handled correctly.

To Reproduce

tool := genkit.DefineTool(....)

prompt := genkit.DefinePrompt(
  g,
  "prompt",
  ai.WithPrompt("my prompt"),
  ai.WithInputType(Input{}),
  ai.WithOutputType(Output{}),
  ai.WithTools(tool),
)

prompt.Execute(ctx, ai.WithInput(in))

Expected behavior
"toolRequests" should be handled correctly when used in combination with ai.WithOutputType

Screenshots
If applicable, add screenshots to help explain your problem.

Runtime (please complete the following information):

• OS: MacOS
• Version 15.7

** Go version

• 1.25.1

Additional context
Add any other context about the problem here.

[apascal07]

Hi there, can I ask which model you are using? Many models don't support constrained output in combination with tool use (since tool use is its own type of constrained output).

[stringl1l1l1l]

I’ve tracked the issue and found the root cause: when using ai.WithTools, ai.GenerateWithRequest may return a tool request message with PartToolRequest, which contains no PartText. With ai.WithOutputType enabled, this wil be formatted by JSON formatter, but my recent formatter changes (#3573) didn’t account for the case where no text parts exist, leading to unnecessary JSON validation. That was my mistake, sorry about that!

By the way, the timing of tool calls from the LLM is unpredictable and confusing. If I want a constrained output type using ai.WithOutputType, I don't want an unexpected tool request. So it would be very useful to have an option to control whether the LLM is allowed to trigger tool calls automatically, since ai.WithReturnToolRequests(true) doesn’t solve this.

@apascal07

[apascal07]

By the way, the timing of tool calls from the LLM is unpredictable and confusing. If I want a constrained output type using ai.WithOutputType, I don't want an unexpected tool request. So it would be very useful to have an option to control whether the LLM is allowed to trigger tool calls automatically, since ai.WithReturnToolRequests(true) doesn’t solve this.

If you are expecting a certain output type then you shouldn't pass in tools (i.e. ai.WithTools(...)) as these become fair game for the LLM to call. Can you clarify what you mean that ai.WithReturnToolRequests(true) doesn't adequately solve not calling a tool automatically?

[stringl1l1l1l]

If you are expecting a certain output type then you shouldn't pass in tools (i.e. ai.WithTools(...)) as these become fair game for the LLM to call. Can you clarify what you mean that ai.WithReturnToolRequests(true) doesn't adequately solve not calling a tool automatically?

​I see, I misunderstood. I thought ai.WithReturnToolRequests(false) prevented the LLM from generating a tool request, but I now realize it just stops the automatic execution of that request.

​My goal is to have the LLM return my customized tool request so I can execute it manually. I'd be willing to try the automatic tool execution, but it seems this functionality isn't supported by all LLMs yet.

[apascal07]

​My goal is to have the LLM return my customized tool request so I can execute it manually. I'd be willing to try the automatic tool execution, but it seems this functionality isn't supported by all LLMs yet.

Yes, that's exactly what ai.WithReturnToolRequests(true) does. It returns the model response (with the ToolRequest part) so that you can handle it as you wish instead of entering an automatic tool-call-and-return-to-model loop.

[janishorsts]

I wanted to report the same issue, and discovered it has already been reported.

2025/09/25 21:38:43 DEBUG model failed to generate output matching expected schema error="data is not valid JSON: unexpected end of JSON input"
2025/09/25 21:38:43 DEBUG span end name=generate

The following test demonstrates the issue:

type In struct {
	Text string `json:"text"`
}

type Out struct {
	Text string `json:"text"`
}

func TestInvalidJson(t *testing.T) {
	slog.SetLogLoggerLevel(slog.LevelDebug)

	ctx := t.Context()

	g := genkit.Init(ctx,
		genkit.WithPlugins(new(googlegenai.GoogleAI)),
		genkit.WithDefaultModel("googleai/gemini-2.5-flash"),
	)

	getWeather := genkit.DefineTool(g, "getWeather", `Fetches the real-time weather for a city`,
		func(ctx *ai.ToolContext, location struct {
			City string `json:"city"`
		}) (string, error) {
			if location.City == "Riga" {
				return "Sunny, 25°C", nil
			}

			return "Cloudy, 18°C", nil
		},
	)

	prompt := genkit.DefinePrompt(g, "weather",
		ai.WithTools(getWeather),
		ai.WithInputType(In{}),
		ai.WithOutputType(Out{}),
		ai.WithPrompt("{{text}}"),
	)

	res, err := prompt.Execute(ctx, ai.WithInput(In{Text: "What is the weather in Riga?"}))
	if err != nil {
		t.Fatalf("want no error, got '%s'", err)
	}

	var out Out

	err = res.Output(&out)
	if err != nil {
		t.Fatalf("want no error, got '%s'", err)
	}

	t.Log(out)
}