wip: v1 chat model #8742
wip: v1 chat model #8742
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
hntrl
force-pushed
the
hunter/v1-chat-model
branch
from
dc230b5 to
8463f1b
Compare
| * and the run ID. | ||
| */ | ||
| handleChatModelStart?( | ||
| llm: Serialized, | ||
| messages: BaseMessage[][], | ||
| messages: Message[], |
There was a problem hiding this comment.
the inner workings of BaseLanguageModel lets you parse multiple prompts at once, whereas the new chat model class only takes one string of messages
| | SimpleChatModel<Runnable<Message, Output>> | ||
| | SimpleChatModel<Runnable<Message, { raw: Message; parsed: Output }>> { | ||
| return super.withStructuredOutput(schema, config) as | ||
| | SimpleChatModel<Runnable<Message, Output>> | ||
| | SimpleChatModel<Runnable<Message, { raw: Message; parsed: Output }>>; |
There was a problem hiding this comment.
unfortunately I'm not sure if there's a clean solution to avoid this verbose typing
| .bindTools([makeToolDefinition("tool1")]) | ||
| .withStructuredOutput({ | ||
| type: "object", | ||
| properties: {}, | ||
| required: [], | ||
| }) | ||
| .bindTools([makeToolDefinition("tool2")]) | ||
| .withConfig({ | ||
| tool_choice: "any", | ||
| }) | ||
| .withStructuredOutput({ | ||
| type: "object", | ||
| properties: {}, | ||
| required: [], | ||
| }) | ||
| .bindTools([makeToolDefinition("tool3")]); |
There was a problem hiding this comment.
would expect that this composition means:
- we keep tools 1-3
tool_choicewill be any- we only use the second withStructuredOutput
| constructor(protected params: BaseChatModelParams) { | ||
| super(params); | ||
| this.verbose = params.verbose ?? false; | ||
| this.callbacks = params.callbacks; | ||
| this.tags = params.tags ?? []; | ||
| this.metadata = params.metadata ?? {}; | ||
| this.cache = iife(() => { | ||
| if (typeof params.cache === "object") { | ||
| return params.cache; | ||
| } else if (params.cache) { | ||
| return InMemoryCacheV1.global(); | ||
| } | ||
| return undefined; | ||
| }); | ||
| this.caller = new AsyncCaller(params ?? {}); | ||
| } |
There was a problem hiding this comment.
this constructor is largely just taking after BaseLangChain
|
|
||
| export abstract class BaseChatModelV1< | ||
| CallOptions extends BaseChatModelCallOptions = BaseChatModelCallOptions, | ||
| TOutputParser extends ChatModelOutputParser | undefined = undefined |
There was a problem hiding this comment.
TOutputParser actually determines whats on the output end of the runnable -- if we want structured output out we use whats defined on the output of the passed in parser, if its undefined we use Message
There was a problem hiding this comment.
this has a naming conflict with non-chat model output parsers, so am open to ideas on what to name this if we stick w/ this
| if (this.outputParser === undefined) { | ||
| yield chunk as InferChatModelOutput<TOutputParser>; | ||
| } else { | ||
| yield this.outputParser.invoke( | ||
| chunk, | ||
| callOptions | ||
| ) as InferChatModelOutput<TOutputParser>; | ||
| } |
There was a problem hiding this comment.
unclear on what the semantics should be for parsing chunks
- run the output parser on each chunk (what this is currently doing)
- only run the output parser on the aggregated chunk
| return instance.withOutputParser( | ||
| toolMessageParser.withConfig({ | ||
| runName: "StructuredOutput", | ||
| }) | ||
| ); |
There was a problem hiding this comment.
instead of returning a runnable, we return a reconstructed instance that has a different outputParser
hntrl
force-pushed
the
v1
branch
2 times, most recently
from
98250ec to
1c42611
Compare
some highlights from this iteration of the
BaseChatModelclass:BaseChatModelnow inherits fromRunnableinstead ofBaseLanguageModelMessageobjects as the output of the "chat model runnable" instead of the legacyGenerationobjecthandleChatModelXinstead ofhandleLLMX)Messageinstead ofGenerationobjects in eventsBaseCachethat usesMessageobjects instead ofGenerationobjectsMessage's to later enable replaying stream chunksRunnableBindingwhen callingwithStructuredOutput,bindTools, orwithConfig, we instead create a new instance of the current class by callingthis.constructorand modifying an internaldefaultOptionsproperty that gets reusedwithStructuredOutputspecifically, we're retaining runnable semantics by pinning the existing runnable chain to a.outputParserproperty that gets used ininvokeand_streamIteratorwithStructuredOutputandbindToolstogether_generate->generategenerate->invoke_streamResponseChunks->streamResponseChunksBaseLanguageModelvery wip, I don't expect CI to pass