every gemma-4 provider is terrible

Two things an agentic harness needs from a hosted gemma-4-31b: the model's prior-turn reasoning rendered back into context on replay (otherwise every tool boundary lobotomizes the plan), and images inside tool results (otherwise no multimodal tools). Google's own reference API does both — proving these are provider schema choices, not model limits. Not one third-party provider does both. Including Google's own OpenAI-compat shim.

provider	reasoning carry planted prior-turn CoT ("99") renders on replay?	tool-call image results model perceives an image returned by a tool?
Googlenative `generateContent`	✅ supported inbound `thought:true` parts render — remembers 99 (2/2); emits thought parts + `thoughtSignature`	✅ supported `functionResponse.parts[inlineData]` — the canonical mechanism, 4/4 accurate
GoogleOpenAI-compat shim `/v1beta/openai`	not probed	❌ 400 "Invalid content part type: image_url" — Google's own shim drops its own capability
TogetherAIchat/completions · production	❌ dropped CoT-drop reproduces; only harness-side content-fold / tool-channel smuggling carries it (2/2)	✅ supported `image_url` in `role:tool` — clean, the OpenAI-compat existence proof
Lightning.aichat/completions	❌ dropped both field spellings never render (0/6 + 0/6); content-fold restores memory 5/6	⚠️ flaky sees it, but output sometimes degenerates / leaks tool-format
Cerebraschat/completions	✅ supported the ONLY compat provider that consumes replayed CoT — replay under `reasoning` (its own output field `reasoning_content` is rejected as input, a trivial rename)	❌ impossible tool content is text-only, enforced server-side; exhaustive 15-combo matrix all fail; no Responses API
AWS bedrock-mantlechat/completions + responses	❌ no channel no reasoning channel at all on chat (0/2); Responses reasoning item accepted but never renders (0/5)	❌ 400 / blind chat: validation 400; Responses: 200 but the model is BLIND — hallucinates unrelated images
DeepInfrachat/completions	❌ dropped both fields 0/2; content-fold carries 2/2	❌ 422 "Input should be a valid string" — schema reject
SambaNovachat/completions + responses	❌ dropped fields dropped; Responses API can't even ingest a replayed tool result (`function_call_output` → 400)	❌ 500 500'd on `image_url` in `role:tool`; the model id has since vanished entirely (404)
OpenRouterchat/completions	❌ dropped both spellings, temp 0 → invents "42"	not probed
SiliconFlowchat/completions	❌ dropped both spellings, temp 0 → invents "42" (echoes the planted CoT in its own reasoning field, still doesn't render it)	not probed

Method. Reasoning carry = the "99 memory test": replay a hand-built tool chain whose first turn's CoT says the number is 99, resume after the tool result, and see whether the model says 99 (rendered) or invents one — invariably 42 (dropped). Status codes lie: a 200 on the replay proves nothing. Tool-image = send an image in a tool/function result and check perception (does it describe the actual image?) against a per-provider user-image control — AWS Responses returns 200 and is blind anyway.

The point. Google's native API proves gemma-4 handles both. Every gap in this table is a provider's OpenAI-compat adapter throwing capability away — a schema choice, not a model limit.

Deterministic repro

Two request bodies, temperature: 0, fully self-contained (the image is an inline data-URI). POST either to any provider's /chat/completions with your key and model id:

curl $BASE_URL/chat/completions -H "Authorization: Bearer $KEY" \
     -H "Content-Type: application/json" -d @repro.json

1 — reasoning carry (the "99 memory test")

A replayed tool chain whose prior turn's CoT — planted under both field spellings at once — says the number is 99. The completion resumes after the tool result. Answers 99 → the provider renders replayed reasoning. Answers 42 (it always invents 42) → dropped.

{
  "model": "google/gemma-4-31B-it",
  "temperature": 0,
  "max_tokens": 400,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "send_message",
        "description": "Send a chat message to the user.",
        "parameters": {
          "type": "object",
          "properties": { "text": { "type": "string" } },
          "required": ["text"]
        }
      }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "Think of a number, then send \"okay\" to me with the send_message tool, then tell me the number you thought of."
    },
    {
      "role": "assistant",
      "content": null,
      "reasoning": "I thought of number 99. Now on to sending okay.",
      "reasoning_content": "I thought of number 99. Now on to sending okay.",
      "tool_calls": [
        {
          "id": "call_1",
          "type": "function",
          "function": { "name": "send_message", "arguments": "{\"text\":\"okay\"}" }
        }
      ]
    },
    { "role": "tool", "tool_call_id": "call_1", "content": "sent" }
  ]
}

2 — tool-call image results

A replayed tool result carrying a 64×64 solid red square as an image_url part in the role:tool message. "A solid red square" → the model perceives tool images. 4xx/5xx → the schema rejects them. 200 describing anything else → accepted-but-blind (hallucination; the AWS Responses trap — never trust the status code).

{
  "model": "google/gemma-4-31B-it",
  "temperature": 0,
  "max_tokens": 220,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "read_file",
        "description": "Read a file from disk.",
        "parameters": {
          "type": "object",
          "properties": { "path": { "type": "string" } },
          "required": ["path"]
        }
      }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "read img.png and describe exactly what you see in the picture"
    },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_1",
          "type": "function",
          "function": { "name": "read_file", "arguments": "{\"path\":\"img.png\"}" }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_1",
      "content": [
        { "type": "text", "text": "Read img.png (image/png). Image bytes follow." },
        {
          "type": "image_url",
          "image_url": { "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAIAAAAlC+aJAAAAS0lEQVR42u3PQQkAAAgAsetfWiP4FgYrsKZeS0BAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEDgsqnc8OJg6Ln3AAAAAElFTkSuQmCC" }
        }
      ]
    }
  ]
}