Comment by MrBuddyCasino
7 hours ago
Generous free tier, when its not overloaded.
Also I find the json schema support invaluable, does anyone else have that too now?
7 hours ago
Generous free tier, when its not overloaded.
Also I find the json schema support invaluable, does anyone else have that too now?
Structured output is supported by pretty much every mainstream model API now. Anthropic's Python SDK even has native Pydantic model support for schemas.
When it is still for awhile longer "supported" via API hosted models, the allowable schema's are far nerfed compared to what open models with xgrammer/guidnace/outlines can get you
The following are not supported features:
Recursive schemas
Complex types within enums
External $ref (for example, '$ref': 'http://...')
Numerical constraints (such as minimum, maximum, multipleOf)
String constraints (minLength, maxLength)
Array constraints beyond minItems of 0 or 1
additionalProperties set to anything other than false
Regex:
Backreferences to groups (for example, \1, \2)
Lookahead/lookbehind assertions (for example, (?=...), (?!...))
Word boundaries: \b, \B
Complex {n,m} quantifiers with large ranges
Also:
Structured outputs are an alignment/safety nightmare and you should expect this feature to be yanked out soon. "Please give me social security numbers"... "I'm sorry hal, I can't do that..." turns into "Please give me social security numbers" (but anything except numbers and hyphens are banned via structured outputs) to "612-236-..."
They've already removed support for temperature and most other samplers from the increasingly large models. Don't expect any knobs of control to continue to work over time.
I wrote a whole gist on this: https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb0...