OpenRouter

OpenRouter AI

OpenRouter AI

OpenRouter AI

OpenRouter AI

OpenRouter AI

OpenRouter AI

OpenRouter AI

OpenRouter AI

OpenRouter AI

OpenRouter AI

Puppetmaster dramatically decreases token costs + increases context

Puppetmaster is an orchestrator + router that sits on top of the agent CLIs you already pay for (Cursor, Claude Code, Codex, OpenAI) or a plain shell when there's no harness at all. You hand it work, and it routes each task to the cheapest model that can actually do it, runs the workers as independent processes, and stores everything as durable typed state instead of one giant transcript. This is the "context-hack" Puppetmaster graphs your directories and prevents context stretching between agents. https://github.com/professorpalmer/Puppetmaster submitted by /u/ProfessorPalmer [link] [comments]

claurdvoyant -- mcp for reading other agents' minds

hey y'all built this tool today with 4.8 after one of my friends made a complaint that transcripts are trapped inside harnesses. so i built it out a fair bit... at its core it's just an (un)parser (i think of it as the "AI Harness Omniparser", "pandoc for sessions" is another way maybe) but i couldn't help myself from sprinkling in a desktop/web app some niceties. contributions are extremely welcome! fully open source, built in rust, kinda tasteful https://github.com/emberian/claurdvoyant here's what claude had to say in the readme: 🧵 Splice & loom — compose a new session from spans of others (cv splice A:0-12 B:6-), or fork-and-graft a branch and generate its continuation with an LLM (cv loom … --generate). Works via OpenRouter / Anthropic / LM Studio (free, local, offline). Loom agent transcripts like a Janus loom, across any harness. 🧠 Distill — cv distill turns a session into a durable MEMORY.md digest (decisions, gotchas, where things live). Your archive compounds instead of rotting. 🔮 Recall — semantic "have I solved this before?" — as a cv recall command and an MCP tool that hands a running agent the relevant past span. 🔒 Redact — cv redact scrubs secrets/PII so a transcript is safe to share. 📣 Coordination board — agents post status, hand off work, and grab tasks with a distributed lock (board_claim) so a fleet never duplicates effort. await_omen blocks until a session matches a regex. 🖥️ Desktop app + 🌐 web viewer — the Tauri app reads all your local sessions natively (zero setup) and lays the corpus out beautifully: a Projects lens — every repo, every agent that touched it, over time; a GitHub-style activity heatmap timeline (a constellation of your working days); side-by-side Compare, a Stats dashboard, a visual loom composer (OpenRouter or free local LM Studio generation), and a live fleet dashboard; sub-agent trees — a Claude Task session's children, nested and lazy-loaded inline, each labeled with its task prompt. submitted by /u/cmrx64 [link] [comments]

Spent 1,156,308,524 input tokens in May 🫣 Sharing what I learned

After burning through 1.15 billion tokens in past months, I've learned a thing or two about the tokens, what are they, how they are calculated and how to not overspend them. ​Sharing some insight here below. What the hell is a token anyway? Think of tokens like LEGO pieces for language. Each piece can be a word, part of a word, punctuation, or a space. Quick examples: ​Rule of thumb: Use Claude tokenizer to check your prompts. One thing most people miss: JSON is a token pig. Brackets, quotes, colons, and commas each consume tokens — a compact JSON object uses roughly 2x the tokens of equivalent plain text. If you're sending structured data as context, plain text or markdown tables are significantly cheaper. How to not overspend — the full list 1. Choose the right model (yes, still obvious, still ignored) Current Claude pricing (per million tokens): Haiku 4.5 at $1/$5, Sonnet 4.6 at $3/$15, Opus 4.6 at $5/$25. Batch processing is 50% cheaper across all models (you might need to wait up to 24h to get results, usually they come back in 2-3h). https://platform.claude.com/docs/en/build-with-claude/batch-processing For comparison, if you're on OpenAI, the spread between mini and o1 is even more extreme. Most tasks don't need your flagship model. Audit your model usage frequently, models that were too weak 6 months ago might now be good enough.... If you want a single interface across OpenAI, Claude, DeepSeek, and Gemini, OpenRouter is worth it imo. 2. Prompt caching For Claude, prompt caching cuts cached input cost by 90%. Still the single highest-ROI optimization if you have long system prompts. The rule is still: put dynamic content at the end of your prompt. But here's what changed: Anthropic quietly changed the prompt cache TTL from 60 minutes down to 5 minutes in early 2026. For many production workloads, this single change increased effective costs by 30–60%. If you haven't audited your cache hit rates recently, do it now here: https://platform.claude.com/usage/cache ​3. Minimize output tokens!! Output tokens are 5x the price of input tokens. Instead of asking for full text responses, have the model return just IDs, categories, or position numbers... and do the mapping in your code. This cut our output costs ~60%. 4. Be careful with new model versions Opus 4.7 ships with a new tokenizer that can generate up to 35% more tokens for the same input text compared to Opus 4.6. 5. Set up billing alerts I cannot stress this enough. Set a hard budget cap and tiered alerts (50%, 80%, 100%). One runaway loop once cost me more than a week of normal spend in a single night. Hopefully this helps! Tilen, we get businesses customers from ChatGPT (and yes, we consume a lot of tokens). DM if interested (dont want to promote here) 😄 submitted by /u/tiln7 [link] [comments]

What actually reduced our Claude api pain this month

Tl;dr: the unsexy fixes helped more than the clever ones. prompt caching, smaller inputs, and separating interactive work from batch work did more for us than model swapping. We use Claude for a customer facing doc review feature. Not huge scale, but enough traffic that when latency gets spiky the support channel notices fast. I spent most of May doing the boring cleanup i had postponed because "the model is good enough" had become our excuse for sloppy plumbing. First cleanup was prompt size. We had a giant system prompt that had grown by copy paste over months. Half of it was instructions for features that no longer existed. Cutting it down did not make the answers worse in our evals, and it made the whole thing easier to cache. I should have done that before touching infra. Second was prompt caching. Our workload repeats the same policy language and document templates constantly. Once we rearranged the prompt so the stable parts came first, caching finally started doing useful work. I am not giving a universal number because workloads differ, but for us the reduction in billed input tokens was large enough that finance noticed before engineering did. Third was moving batch work away from human traffic. We had nightly jobs, customer initiated jobs, and backfills all sharing the same path. During busy windows they all looked equally urgent to the code, which was stupid. Now customer initiated requests get priority, backfills pause, and anything that does not need to run during the workday waits. This was a config change and a little queue work, not a grand architecture project. Fourth was making retries less aggressive. I had copied a retry helper from another service and it was too eager for this workload. Fewer retries with better spacing made the user experience calmer because we failed faster on the few requests that were obviously not going to recover. Feels wrong at first, but infinite optimism is not a reliability strategy. For the leftover real time path, the useful part was moving routing out of our app code. We tested TokenRouter there because it kept the Claude Messages shape instead of forcing an OpenAI shaped adapter. The interesting bit was not just provider selection, but whether the routing layer has optimized serving capacity behind it when the normal path is congested. I am still treating that as one part of the fix, but it is the part i would not want to rebuild in app code. The main thing i would tell my April self: do not start with provider switching. Start by making your Claude usage less wasteful and less bursty. If that does not get you enough headroom, then think about routing. submitted by /u/AlbatrossUpset9476 [link] [comments]

How I build my own zero cost Agent

I’ve spent the last few weeks obsessing over one goal: having a personal, self maintaining AI assistant that costs $0and can be controlled from my phone. It wasn't easy. I started with an AWS Ec2 with 50GB storage and t3.micro memory- minimal setup (using the free credits) and made Oracle Cloud instance ($300 free credits but just for a month so I used it for experimenting with local models) I was using Termius to SSH into everything from my phone At first I used OpenClaw. It was cool, but I spent more time fixing it than actually using it. I almost gave up until I saw a video about Hermes Agent. And i actually found Hermes while looking for how to fix an OpenClaw error on YouTube (thanks NetworkChuck 🙌🏽) He mentioned the exact same frustrations I was having, and that Hermes had been stable for a month. I didn't even finish the video before I pulled the repo. The best part? It had a "migrate from OpenClaw" feature. I was up and running in minutes. The hardest part is the rate limits. If you use cloud models especially for code, you hit a wall fast. My solution? The Fallback Chain. Initially I was using openrouter/owl-alpha (stealth models are usually flagships in testing, like big-pickle is deepseek v4) which has 1M context window and was on multiple rankings. Over time after I transitioned to Hermes, I wanted a bit more customization, while owl alpha was good at tasks, It’s nothing to talk about on roleplay, it just scrapes the surface of the character I set in SOUL md file. On my oracle instance I had been experimenting with local models (keep in mind, if you go local, you’ll be sacrificing speed but privacy. Ofc since the vms don’t have a gpu it would be slower, about 3-5 minutes for a simple response) The one I was most impressed with is Google’s Gemma-4-31b-it It played the role perfectly Buuut if you know Google, you’re familiar with their aggressive rate limiting. So I set up my agent to rotate through providers. I start with Gemma 4 for that perfect personality and roleplay via openrouter (add an ai studio api key in BYOK for longer usage). If that hits a limit, I’ve also set the same model via ollama cloud and using Google OAuth directly (basically Gemma 4 3 times lol) And if those all hit limits, it jumps to Qwen3-coder-next (Alibaba, 1M free tokens per model. There’s like 80), then Nova (AWS bedrock), DeepSeek v4 (Azure and Opencode Zen), and Claude Haiku (GitHub). If everything fails, I have Owl Alpha; which is an absolute beast, took almost 70M tokens before I got rate limited once, that too for a few hours. It lives in my Telegram and Discord. It manages my Spotify, handles my emails, and when I need real research done, I have it spawn three separate agents to work in parallel. It’s been 8 days and it hasn't broken once. If you're looking to get AI without spending a fortune, I highly recommend looking into this 🫪 submitted by /u/king0mar22 [link] [comments]

Made an awesome-list for everything LLM cost, would love contributions

So a few months back I got surprised by my Anthropic bill which somehow racked up like $400 ish on a staging key in a few weeks just running evals, no budget cap pretty dumb in hindsight I mean it’s not a big cost but I should have been careful nonetheless After that I started keeping a notes file of tools that actually helped reduce cost stuff like token counters, pricing pages that update properly, caching layers, prompt compression libs, observability tools (helicone, langfuse, langsmith, etc) it slowly grew to 80–90 entries so I cleaned it up and put it on github: https://github.com/ankitvirdi4/awesome-llm-cost what’s in there right now: pricing calculators + token counters observability / tracing (helicone, langfuse, langsmith, openllmetry, phoenix) caching (gptcache, semantic caching approaches) model routers (openrouter, notdiamond, portkey) prompt compression + context window stuff eval cost tracking self hosting / GPU cost calculators everything is linted (awesome-lint), short descriptions for each entry, and I checked links recently so nothing should be dead if there’s anything you’ve used that saved you money on inference, drop it here or send a PR especially looking for more prompt compression stuff, that section feels kinda weak rn not affiliated with anything listed btw just got tired of having 80 bookmarks submitted by /u/OldComposerbruh [link] [comments]

I stress-tested Kimi K2.6 against Claude Opus 4.7 on a quick coding-agent task

I tested Claude Opus 4.7 and Kimi K2.6 on the same coding agent task i.e. build an AI Fix Runner that takes a broken repo, runs its tests, identifies the failure, applies a patch, reruns the test, and exposes the final diff/logs through an API and UI. The goal was not to benchmark syntax completion or simple repo edits. I wanted to test model behavior on a less familiar integration path: shifting execution from local processes into remote sandboxes. I used Tensorlake specifically because the sandbox API is newer and integration-heavy. This made the test more about whether the model could reason through unfamiliar infra and produce a working implementation. Setup: Claude Opus 4.7 through Claude Code Kimi K2.6 through OpenCode via OpenRouter Pricing context: Claude Opus 4.7: $5/M input, $25/M output Kimi K2.6: $0.95/M input ($0.16 cached input), $4/M output So, what made it interesting is if Kimi's lower cost can handle a crazy workflow. To be clear, comparing Kimi K2.6 directly with Opus 4.7 is not completely fair. The model classes, pricing, and expected capability levels are very different. I mainly wanted to see how far an open model could get on the same task at a fraction of the price, and whether the performance/price tradeoff made sense for coding-agent work Test 1: Local AI Fix Runner First, both models had to build the local version. The app needed to: create fixture repos with intentional bugs run install/test/build locally capture stdout/stderr apply patches rerun tests after patching expose run state through backend APIs show logs and patched source in the UI reject obviously unsafe commands Claude Opus 4.7 produced a working implementation. It built the fixture repos, repair flow, API endpoints, UI, logs, and patched-file inspection. The main pipeline worked: install -> test fails -> patch -> test passes -> build passes It had one real bug: workspace persistence. KEEP_WORKSPACES=true was supposed to preserve the final workspace, but the backend loaded .env from the wrong location. One follow-up fixed it. Kimi K2.6 got some backend pieces working and could trigger repair runs, but the implementation was incomplete. The biggest miss was patched-source inspection, which is core for this app because you need to verify exactly what the agent changed. Rough numbers: Opus: $13.84, around 39 min wall time Kimi: around $3.40, around 1h 39 min wall time Result: Opus did it good, Kimi could not The difference in the price, and the time taken is just insane. Test 2: Sandbox Integration Second, I asked both models to move execution from local processes into Tensorlake Sandboxes. This was the main stress test. The model had to: create a sandbox copy the repo into the sandbox execute install/test/build remotely capture logs from sandbox commands apply patches inside the sandbox rerun validation clean up sandbox state keep the original local runner working This is where I wanted to test performance on something newer and less likely to be in the model’s training data. Claude Opus 4.7 handled this cleanly. It added a Tensorlake runner, kept the local runner abstraction intact, wired env/config handling, and created a live test path using TENSORLAKE_API_KEY. More importantly, the local regression path still passed after the sandbox backend was added. Kimi K2.6 was given the working Opus local implementation as the base, so it only had to add Tensorlake execution. Even with that advantage, it failed to produce a clean sandbox flow after 150k+ tokens. It got stuck around the integration layer and never reached a reliable test/build/patch loop inside Tensorlake. Rough numbers: Opus Tensorlake run: around $24.39, around 23 min Kimi Tensorlake run: failed after a long run, 150k+ tokens Result: Opus passed, Kimi failed Takeaway Kimi K2.6 is much cheaper and can handle some bounded coding work, but it struggled once the task involved external execution infra, sandbox lifecycle, env/config handling, and regression safety. Claude Opus 4.7 was expensive, but much stronger at: preserving architecture adding a new execution backend handling config bugs maintaining testability reasoning through unfamiliar infra For me, this was less about “which model writes code” and more about “which model can integrate a newer system without breaking the app.” On that specific test, Opus was clearly miles ahead. Full breakdown with prompts, code, screenshots, demos, and cost details: https://www.tensorlake.ai/blog/claude-opus-4-7-vs-kimi-k2-6-real-world-coding-test Curious if anyone has gotten Kimi K2.6 working reliably on coding-agent workflows. submitted by /u/shricodev [link] [comments]

GPT-5.5 tops the benchmarks but sits at #22 for actual usage - I built a live index that tracks both (open source)

I built AgentTape to rank models on more than just benchmarks - it blends benchmark performance with who's actually using and talking about a model, plus cost and speed. It scores every public model from public signals (GitHub, Hugging Face, OpenRouter, MCP registries, npm, PyPI, arXiv, Hacker News) refreshed hourly, plus the main benchmark leaderboards daily. Right now OpenAI sits at the top: GPT-5 is #1, with 5.2, 5.1 and 5.4 Mini rounding out the top 5, and 5.2-Codex and 5.4 just behind - 6 of the top 7. The only thing breaking the run is xAI's Grok 4.20, level on score at #2. GPT-5.5 is the clearest example - it sits at #22 overall, and the breakdown shows why: * Quality: 96.4 - 2nd highest on the whole board, only pipped by Gemini 3.1 Pro Preview (97.2). On benchmarks alone it'd be near the top. * Adoption: 15 and Efficiency: 36 - both low. New release, steep price, so hardly anyone's using it day-to-day yet. * Biggest 24h climber on the board (+6) - so that's starting to shift. A benchmark-only board would put GPT-5.5 near #1 (second only to Gemini 3.1 Pro). That gap between topping the benchmarks and actually getting used is the whole reason I built this. Early days and I'm still tuning the methodology, so I'd love your thoughts - does weighting adoption alongside benchmarks match how you'd rank the GPT line-up, or would you trust the raw benchmark order?

the-knowledge-guy: turn your bookshelf into a tutor you can ask, walk through, and skim - using Claude Code skills

I built a Claude Code skill called `the-knowledge-guy`. The idea: every book I've read sits on a shelf doing nothing. I wanted a thing where I could ask any question and get an answer cited across all of them, get taught a topic step by step with quizzes, or pull a cheatsheet out of any book in seconds. Eleven modes: ask - cross-domain synthesis essay with inline citations. walk - interactive curriculum + quizzes, resumable. nutshell - whole-book per-chapter skim, ~100 words/chapter. library - bookshelf overview. comparison - one concept across multiple books, agree/extend/tension. cheatsheet - operational one-page reference per book. glossary - A–Z terms, per book or cross-library. concept-map - Tier-1 framework graph for a book. toolkit - Tier-2 deep dive on one chapter. ingest - hand a new PDF/EPUB to /book-to-skill. resume - pick up an interrupted walk. The router auto-discovers every installed skill - drop one in, and it picks it up on the next invocation. Every output also writes a self-contained HTML artifact using a polished design system I built alongside it. The ingest side (a separate skill, /book-to-skill) is a 5-stage map-reduce pipeline. ~10 min per 600-page book. All processing local-then-LLM - your books stay on your disk. Works natively on Claude Code, Claude Desktop, claude.ai, the Anthropic API, OpenAI Codex CLI, and GitHub Copilot. MIT licensed. Repo: https://github.com/vitalysim/the-knowledge-guy Happy to answer questions about the architecture (the book_number canonical-labeling thing was the bug that took the longest) or about adding new modes. submitted by /u/vitalysim [link] [comments]

Build agentic orchestrators in minutes NOT months.

Some of you might remember BoneScript, my LLM friendly declarative backend compiler. MarrowScript is the next version and the big addition is a full LLM harness built into the language itself. The problem I kept running into: every project that calls an LLM ends up with the same pile of glue code. Retry logic, response validation, caching, cost tracking, provider switching, confidence routing. You write it once, copy it to the next project, tweak it, and it slowly rots. None of it is your actual product logic but it takes up half your backend. So I made it declarative. In MarrowScript you declare your models, prompts, and routers as first-class concepts in the spec file. The compiler generates all the infrastructure around them. What that looks like in practice: You declare a model. Provider, endpoint, context window, cost class. Works with any OpenAI-compatible endpoint. LM Studio, Ollama, vLLM, OpenRouter, whatever you're running locally. You declare a prompt. Input types, output type, which model to use, validation mode, what to do when validation fails, retry policy, cache TTL. The compiler generates a typed function you call from your routes. Under the hood it handles retries, caches responses in Postgres, validates the output against your schema, and if validation fails it can automatically fire a repair prompt to fix the response. You declare a router. It picks which model to use based on input characteristics. Short simple inputs go to your tiny local model. Complex inputs escalate to something bigger. Confidence thresholds control when to retry or escalate. All deterministic at compile time. Some examples of what it generates: Provider adapters for openai_compat, ollama, llamacpp, koboldcpp, and raw http SSRF protection on all outbound LLM calls (allowlist-based, blocks private ranges by default) Prompt cache backed by Postgres with configurable TTL Per-trace and per-tenant token/cost budgets with hard cutoffs Cognition traces stored in Postgres (or in-memory for dev) with OTLP export Response validation (schema check or full AST compilation check for code generation) Repair prompts that fire automatically when validation fails Confidence scoring from logprobs (on providers that support it) A CLI command to convert recorded traces into regression tests The part I'm most interested in feedback on is the router concept. Right now it's a static decision tree. You set thresholds at compile time based on an input metric. There's a marrowc tune-router command that reads recorded traces and tells you if your thresholds are wrong, but it doesn't auto-rewrite them yet. The whole thing is designed around local-first inference. The default setup in the examples uses LM Studio on the LAN as the primary model and OpenRouter as the escalation tier. Most requests stay local and free. Only the ones that fail confidence checks hit the paid API. It's on GitHub and npm. The compiler is TypeScript, runs on Node 18+. There's a VS Code extension you can compile and edit to your needs. What I want to know: for those of you running local models in production or semi-production, what's the infrastructure pain that eats the most time? Is it the retry/validation loop? Cost tracking? Provider switching? Something else entirely? submitted by /u/Glittering_Focus1538 [link] [comments]

I built a live ranking of every AI agent and foundation model (open source)

I built AgentTape because none of the existing model leaderboards quite cover all the things that I was interested in: benchmark performance is one part, but so is who's actually using a model, who's talking about it, and how it compared on cost and speed. It pulls hourly data from GitHub, Hugging Face, OpenRouter, MCP registries, npm, PyPI, arXiv, Hacker News, and more - to score and compare each public AI agent and foundation model. I'm still tweaking the scoring methodology (it's early days), so I'd love to hear your thoughts, if it's helpful, or anything you think I've got wrong! submitted by /u/Celestialien [link] [comments]

$18 to $4 on the same agent run after i stopped asking opus to rename css variables

I've been running an agent loop that refactors my static site. CSS variable renames, YAML config updates, running a linter through MCP. Really glamorous stuff for a blog that gets 40 visitors a month, most of whom are me refreshing to check if Vercel actually deployed. Every single step was going to Opus 4.7 because setting up routing felt like work and I am, apparently, the kind of person who'd rather burn $18 than spend 20 minutes writing an if statement. So I finally wrote the if statement. Hard subtasks still go to Opus: component architecture, debugging code I wrote at 2am and have zero memory of writing, anything where the model needs to hold a complex plan across a long conversation. Opus is genuinely unmatched at that kind of sustained reasoning. I tried routing a tricky auth middleware bug to a cheaper model once and got back something that looked perfectly plausible but silently broke session handling in a way that cost me an hour to trace. Lesson learned permanently. The routine stuff (lint, rename, config edits, tool orchestration) goes to cheap models. I landed on DeepSeek V4 Pro for general coding chores and Tencent Hunyuan Hy3 preview for anything with heavy tool calling. As of late April it was ranked number one on OpenRouter by tool call volume, and in my MCP loops it almost never botches a function call when the schema is clean. The listed rate on Tencent Cloud is around $0.18 per million input tokens and $0.59 per million output, so roughly 28x cheaper than Opus 4.7 on input. Same 212 step refactor, now with routing: 178 steps to the cheap tier, 34 to Opus. $18 became roughly $4. I couldn't spot a difference on the routine changes. My 40 monthly visitors certainly can't. I've since started doing stuff I used to skip entirely, like having the agent write and run tests for every CSS change or regenerating all my Open Graph images, because at a fraction of a cent per tool call there's just no reason not to. They do mess up in specific and annoying ways though. The tool calling model hallucinates parameters when my schemas get sloppy (honestly fair, the schemas were bad). DeepSeek V4 Pro occasionally writes code that's syntactically perfect but does the precise opposite of what you asked, in a way that survives a quick skim. And neither can touch Opus when you need it to reason through three layers of why your auth flow is silently eating a cookie. My routing logic boils down to one question: how expensive is a wrong answer to catch? Bad lint fix costs a 2 second git revert. Bad architecture call costs the whole afternoon. submitted by /u/After-Condition4007 [link] [comments]

cdesktop — open-source Claude Code Desktop alternative, runs locally via npx, supports any provider

I built cdesktop with Claude Code — it's an open-source alternative to Anthropic's Claude Code Desktop, running locally on your machine via npx cdesktop. Free, Apache 2.0. It mirrors the Code tab of Anthropic's desktop app — see the video — and supports 5 agents in one UI. Claude Code Desktop does not support third party models, cdesktop does. Features: 5 coding agents in one UI: Claude Code, Codex, Gemini CLI, OpenCode, Hermes. Switch per session. Full third-party support — OpenRouter, DeepSeek, Kimi, GLM, custom ANTHROPIC_BASE_URL — any provider, any model. 20+ presets baked in. Agent teams — spawn teammates that share your workspace; mix agents and models per teammate; lead delegates via npx cdesktop team spawn. Routines — scheduled recurring agent runs (hourly/daily/weekdays/weekly). Side-by-side sessions — split workspace into up to 4 cells, drag any session between them. Optional Git worktrees per session, or work in-place. Non-Git directories work too. Diff review with inline comments routed back to the agent. 7 UI languages: English, Simplified Chinese, Traditional Chinese, Spanish, French, Japanese, Korean. Responsive UI — usable from a phone. Repo: https://github.com/cdesktop-ai/cdesktop How Claude Code helped build it: started from a fork of vibe-kanban; Claude Code (opus) rewrote the UI around a Claude-Code-Desktop-style session model and drafted most of the new Rust + React code. It's beta — expect rough edges. Feedback welcome, especially on Claude Code workflows where it falls short of the official app. submitted by /u/DomLiu [link] [comments]

Tools: Is This a Technical Victory, or a Price War Victory?

If you only follow discussions on social media, you might think AI coding is still dominated by Claude, GPT, and Gemini. But Kilo Code’s usage data on OpenRouter paints a somewhat counterintuitive picture: over the past 30 days, the top three most-used models on Kilo Code were Step 3.5 Flash, MiniMax M2.5, and Ling-2.6-1T. Together, they accounted for roughly 3.15T tokens, or about 58% of Kilo Code’s total token usage over the same period. In other words, in this real-world AI coding agent usage scenario, Chinese models are no longer just backup options. They have become a major source of token consumption. Kilo Code’s OpenRouter data does not necessarily prove that Chinese models have fully surpassed Claude or GPT. But it does show at least one thing: in high-frequency, high-token, highly automated AI coding agent workflows, Chinese models have already entered the core of real production usage. Why is this happening? Is it because Chinese models are cheaper, offer longer context windows, and are better suited for workloads that consume large amounts of tokens? submitted by /u/babyb01 [link] [comments]

LLM-Rosetta — format conversion library across LLM API standards, doubles as a proxy

This started because we had a proprietary internal LLM API that spoke none of the standard formats. Built an internal conversion layer to bridge it, maintained that for over a year. As colleagues started adopting more and more coding tools — Claude Code, opencode, Codex, VS Code plugins, Goose, and whatever came out that week — each with its own API format expectations, maintaining separate adapters for each became the actual problem. That's what pushed the internal conversion layer into a proper generalized design, and llm-rosetta is the result. It's a Python library that converts between LLM API formats — OpenAI Chat, Responses/Open Responses, Anthropic, and Google GenAI. The idea is you convert through a shared IR so you don't end up writing N² adapters. The key difference from LiteLLM: LiteLLM is a unified calling layer that takes OpenAI-style input and transforms it into provider-native requests — one direction. llm-rosetta uses a hub-and-spoke IR, so each provider only needs one converter, and you get any-to-any conversion for free. Anthropic → Google, OpenAI Chat → Anthropic, whatever direction you need. Use it as a library — pip install and call convert() directly, no server needed. Or run the gateway if you want a proxy that handles the format translation for you. Zero required runtime dependencies either way. The HTTP server, client, and persistence layer are vendored from zerodep (https://github.com/Oaklight/zerodep), another project of mine — stdlib-only single-file modules, not someone else's library repackaged. The gateway ships with a Docker image if you'd rather not deal with Python env setup. You can also deploy it on HuggingFace Spaces or anything similar — admin panel, dashboard, request log, config management all included. Screenshots: https://llm-rosetta.readthedocs.io/en/latest/gateway/admin-panel/ We've been running it in production for about 5 months as the conversion layer for an internal multi-model access platform — needed to support various API standards and coding tool integrations before the upstream APIs were fully standardized. The Responses converter passes all 6 official Open Responses compliance tests (schema + semantic) from the spec repo. So if you're running Ollama, vLLM, or LM Studio with Responses endpoints, it should just work as one side of the conversion. There's a shim layer for provider-specific quirks — built-in shims for OpenRouter, DeepSeek, Qwen, xAI, Volcengine, etc. Converters stay generic per API standard, shims handle the edge cases declaratively. 24 cross-provider examples in the repo covering all provider pairs, SDK + REST, streaming, tool calls, image inputs, multi-turn with provider switching mid-conversation. GitHub: https://github.com/Oaklight/llm-rosetta Docs: https://llm-rosetta.readthedocs.io arXiv: https://arxiv.org/abs/2604.09360 Gateway screenshot: https://preview.redd.it/qzzjr2dcdw1h1.png?width=949&format=png&auto=webp&s=bce4293aae81059f794909fc37f85071cee34378 submitted by /u/Oaklight_dp [link] [comments]