Simon Willison

So you want to use OpenRouter?

Read the original on Simon Willison →

OpenRouter advertises automatic fallback and cost‑effective routing to the best backend provider for a single API endpoint. However, Mohamed Moustafa highlights that different providers use varied serving software, optimizations, and capabilities, leading to inconsistent behavior across the same endpoint. Users can mitigate this by specifying a provider with the provider.only option and checking available providers via the /endpoints method.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Simon Willison.

Simon Willison
Oct 6

EmbeddingGemma 2

Simon Willison comments on EmbeddingGemma 2, noting its Apache 2.0 license and expressing preference for open‑weight models over proprietary, hosted‑only options. He argues that embedding models are often used to generate and store large numbers of vectors, and a closed model could force costly re‑embedding if the vendor discontinues service. Willison prefers a hosted solution that allows him to switch to the open‑weight version if needed.

Simon Willison
Oct 6

llm-openai-decisions 0.1a0

Simon Willison announces the release of the llm-openai-decisions 0.1a0 plugin, which interfaces with OpenAI’s new Jev-style Decisions API. The plugin, inspired by llm-typesafe, supports image and text input and offers the same three question types (yes/no, choices, scores) as Jev, with pricing at 10¢ per million input tokens. Installation is simple via `llm install llm-openai-decisions`, and an example query demonstrates image-based evaluation.

Simon Willison
Sep 22

llm 0.36

The release of llm 0.36 introduces new OpenAI models gpt-6-sol and gpt-6-luna, and adds support for model plugins to declare that they do not support conversations via supports_conversation = False. When such models receive assistant or tool history, llm raises a ConversationNotSupported error and the chat interface rejects them before starting a session. Additional changes include wrapping reasoning traces in Markdown output with <details> tags and bug fixes from five contributors.

Simon Willison
Aug 22

llm 0.33

The release of llm 0.33 introduces several key updates: it upgrades to the OpenAI Python library 3.x and switches the HTTP client from httpx to httpx2, adds comprehensive key handling for embedding functions, allows repeated template usage to combine configurations, and adds a reasoning_summary option for reasoning-capable responses. These changes improve compatibility, flexibility, and usability of the llm tool.

Simon Willison
Oct 29

API credits benefit for eligible ChatGPT subscriptions

Anthropic and OpenAI have introduced matching API credit benefits for their premium ChatGPT subscription plans. Subscribers paying $100, $200, or $500 per month receive an equivalent amount of expiring credits that can be applied to their respective APIs. The author, who holds subscriptions with both services, notes this allows for immediate use in public-facing demos without additional cost.