Skip to content

Conversation

stringl1l1l1l
Copy link
Contributor

Fixes #3569

This PR fixes the "unexpected end of JSON input" error that occurs when combining streaming output (ai.WithStreaming) with structured response parsing (resp.Output(&struct)) in Firebase Genkit.

Problem

When using streaming responses with structured output formats (JSON, JSONL, Array, Enum), the content gets fragmented across multiple ai.Part objects. The format handlers were processing each part individually, expecting complete format-compliant content, but receiving only fragments. This caused parsing failures when trying to validate incomplete JSON/JSONL/Array/Enum values.

Root Cause

In streaming scenarios, model responses are split into multiple parts:

// Streaming response parts
Content: []*Part{
    &Part{Text: `{"name"`},      // Fragment 1
    &Part{Text: `": "John", `},  // Fragment 2  
    &Part{Text: `"age": 30}`},   // Fragment 3
}

Each format handler was validating individual parts, but fragments like {"name" are not valid JSON.

Solution

Updated all format handlers to:

  1. Accumulate text content: Collect all text parts before processing
  2. Process combined content: Parse/validate the complete accumulated text
  3. Preserve non-text parts: Maintain tool calls and other non-text content
  4. Maintain backward compatibility: Single-part responses continue to work unchanged

Changes Made

  • format_json.go: Accumulate text parts before JSON extraction and validation
  • format_jsonl.go: Combine text before extracting JSONL lines
  • format_array.go: Merge text content before processing JSON objects
  • format_enum.go: Concatenate text before enum validation
  • format_text.go: No changes needed (already works correctly)

Testing

Added comprehensive streaming test cases covering:

  • ✅ Multi-part JSON parsing with schema validation
  • ✅ Multi-part JSONL line extraction
  • ✅ Multi-part Array object processing
  • ✅ Multi-part Enum value validation
  • ✅ Mixed content scenarios (text + tool calls)
  • ✅ Error handling for invalid streaming content

Impact

  • Fixes streaming + structured output combination
  • Maintains backward compatibility
  • Preserves all existing functionality
  • No breaking changes

Checklist

  • PR title follows Conventional Commits
  • Tested with comprehensive unit tests covering streaming scenarios
  • All existing tests pass
  • No documentation changes required (internal fix)

@apascal07
Copy link
Collaborator

Thank you for the PR, looks great! Will merge as soon as the sole comment is resolved.

@apascal07
Copy link
Collaborator

@apascal07 apascal07 merged commit 9838d17 into firebase:main Sep 18, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Go] Streaming breaks structured JSON output parsing
2 participants