Repository navigation
Conversation
includeSchemaInPrompt was honored by the local models and Foundation Models, but the HTTP models only sent the schema to the server. A server that ignores it, such as an OpenAI-compatible API or one of Ollama's cloud models, then gets a model that answers in prose. Ollama, OpenAI (both variants), Open Responses, Anthropic and Gemini now also say the schema after the prompt being answered, and still send it to the server. Said there, next to what's asked, a model follows it even after rounds of tool calls, where some overlook a system message. Free text has no schema to say. The three local models' copies of the schema prompt become one, GenerationSchema.schemaPrompt(), which says the inlined schema.
A recursive schema can't be inlined, so the shared schema prompt said only "Respond with valid JSON only." It now says the schema as it's encoded, with its definitions, as the local models' helpers did.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #369. HTTP models (Ollama, OpenAI, Open Responses, Anthropic, Gemini) now honor
includeSchemaInPrompt: for a response that isn'tString, the schema instructions are added to the last prompt entry of the request's transcript. The schema is still sent to the server as before.The private
schemaPrompt(for:)copies in MLX, llama.cpp and Core ML are now one sharedGenerationSchema.schemaPrompt(). It says the inlined schema, or, when the schema can't be inlined, as with a recursive one, the schema as it's encoded, with its definitions.Tested for every HTTP model, streamed and not, after a tool round and with
includeSchemaInPrompt: false, plus a recursive schema's prompt. The plain-text fallback and the retry from the issue's discussion aren't part of this.