What inference is, and why it became the cost problem
Using an AI model splits into two phases. Training builds the weights by pushing vast data through the model; you do it once. Inference is the moment you type a prompt and get an answer.
Training happens once, but inference repeats forever, multiplied by users and requests. The longer a service runs, the larger inference looms in total cost. Agents made this worse: a single request no longer means a single inference. The system calls a tool, reads the result, decides again, dozens of times, and latency compounds at every step.
A second problem sits on top. When you reach intelligence through a closed model's API, internal documents and code leave your boundary with every request. That is tolerable for one person and hard to govern once an entire organization runs agents.
This week's news offers three different answers to those two problems: build your own chip (cost), use models whose weights are open (optionality), or run the whole thing on your own machine (data).
OpenAI Jalapeño: what changes when a model company builds its own chip
OpenAI published the first measured results from Jalapeño, its first custom inference chip. It is inference-only; training new models is not its job.
Measurement ran on InferenceX, a public benchmark from SemiAnalysis. The notable choice is the models used: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, all open-weight models anyone can download and run. Numbers from a closed in-house model would be unverifiable; numbers from public models let other companies put the same model on different silicon and compare.
Across all three, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems. On Kimi, the largest model tested, that was roughly 1.5 times higher performance per watt and 3.4 times lower end-to-end latency. The chip is rated at 700 watts while measured sustained power stayed at or below 550 watts. OpenAI says the advantage widened on its own frontier models internally, which outsiders cannot verify.
Design to tapeout took nine months, with OpenAI models used in the loop. Deployment inside OpenAI's own infrastructure begins by the end of the year; a second generation is deep in development and a third is taking shape.
One sentence in the same post matters as much as the numbers: OpenAI will continue to widely deploy NVIDIA and other partner accelerators for both training and inference. This is not replacement but an added option, and having the option creates pricing leverage.
What open-weight models are, and what shipped this week
An open-weight model publishes the trained weight files so anyone can download and run it. A closed model runs only on the vendor's servers and you receive results through an API. The difference is where the model can run: with the weights in hand, you choose the cloud, or no cloud at all.
Two arrived this week. Z.ai's GLM-5.3-Flash is the first natively multimodal model in the GLM-5 series, published on Hugging Face under the MIT license. Z.ai says it outperforms GLM-5.2 across benchmarks and real workloads at one-tenth the price, and approaches Claude Opus 4.8 on coding and agentic benchmarks.
The other is Qwen3.8-Flash-Next, whose model card describes it as an experimental preview of the architecture that will underpin Qwen4 — the next generation's structure, opened early.
Both target efficiency rather than raw capability: the same intelligence for less compute, which is what makes running outside someone else's cloud possible.
What active parameters mean: 320B stored, 18B switched on
GLM-5.3-Flash is described as 320B total parameters with 18B active. That reads like a contradiction until you know the Mixture of Experts structure.
The model holds many expert modules, and processing one token switches on only some of them. You must store all 320B, but only 18B participate in any given computation. Memory looks like a large model; compute and cost look like a small one.
Qwen3.8-Flash-Next pushes the same idea: 125B parameters with 6B activated, plus a separate 51B of n-gram embeddings and 4B of MTP. The separate embedding axis is the interesting part — it scales parameters along a path that costs less compute and is easier to offload, which helps on memory-constrained accelerators.
Attention changed too. Gated DeltaNet is paired with Qwen Sparse Attention, which selects at the micro-block level rather than picking individual tokens, cutting long-context latency. GLM-5.3-Flash attacks the same problem by mixing sparse and linear attention for the first time in the GLM series.
Different structures, one goal: make long context cheaper. Agents cannot avoid long context, so that is the expensive seat right now.
Agents come down to your machine: Perplexity Portable Computer
Perplexity's Portable Computer is a local-first agent. The model, the harness, the conversation, and the trajectory all live on the user's machine. Only work that needs the outside world — web search, connectors, escalation to a stronger advisor model in the cloud — crosses the boundary, and always with the user's approval.
A harness here is the program around the model: it attaches tools, runs the loop deciding what to do next, and assembles context. If the model is the engine, the harness is the chassis.
Perplexity's claim is that local models need local harnesses. General-purpose harnesses assume a frontier model that absorbs long contexts and navigates a broad tool surface; smaller models are less reliable under those demands.
Four design principles are concrete. First, context economy: even though on-device models advertise 260K-token windows, they empirically begin to struggle past 100K, so the core harness stays minimal and everything else loads and unloads as on-demand skills. Second, connectors become compact command-line tools rather than MCP servers, whose long tool definitions eat context. Third, self-verification adds steps but substantially narrows the gap to frontier models. Fourth, sandboxing is mandatory: if the sandbox is unavailable, the harness disables itself rather than running tools unsandboxed.
On Perplexity's own benchmark of 53 everyday knowledge-work tasks, Qwen 3.8 27B on an NVIDIA DGX Spark scored 82.6%, versus 77.6% for Pi and 74.0% for Hermes under the same conditions; their post-trained model reached 85.4%. Read a vendor benchmark with the usual caution, but the direction is the point: the score moved by fixing the harness, not by enlarging the model.
What Google shipped the same week: video control, transcription accuracy, checkout in search
Gemini Omni 1.1 Flash is a video model update in which everything added is control. Extend a scene up to 40 seconds, set start and end frames so the model builds the transition between them, attach up to three seconds of reference video for character consistency, and upscale finished work to 1080p or 4K.
The practical change is the 360p draft: up to 60% faster and one third the cost of 720p. Explore cheaply, then render only the chosen take at 4K. That is workflow cost design, not image quality.
Gemini 3.5 Transcribe reports, as measured by Artificial Analysis, a word error rate of 4.0% streaming and 2.6% non-streaming, with time to final transcription improved by 70% over the previous Chirp 3 model. The handling matters more than the numbers: it folds in self-corrections such as "let's meet Tuesday — no, Wednesday," strips filler words, and auto-formats. It detects and transcribes over 85 languages and attributes up to three speakers with timestamps on pre-recorded audio.
In Search, AI Mode gained three travel capabilities: flight price tracking set up mid-conversation against the latest fares from more than 300 airline and travel-site partners, available in over 180 countries; points and miles pricing for flights and hotels; and hotel booking where "Continue on Google" leads to room selection, cancellation terms, and checkout via Google Pay. The hotel or booking platform remains the merchant of record and handles customer service. It is rolling out in the U.S. in English first.
The common thread is not model benchmarks but how output slots into an actual workflow.
Money, robots, and the development environment leaving its desk
Stability AI announced on August 25 that it raised $76M in a Series B, bringing total funding to $232M. The investor list is the story: Sony Music Group, Universal Music Group, and Warner Music Group joined together, along with Electronic Arts from games, plus AMD Ventures and Pacific Alliance Ventures. The music industry is not only litigating against generative AI; it is also buying in.
In robotics, Skild AI released its foundation model S1 under four conditions: unseen tasks, ten-minute horizons, one video prompt, no post-training. Its diagnosis is that robotics has been stuck in the BERT era — teaching a robust new task still meant collecting tens to hundreds of hours of data in deployment conditions and fine-tuning a specialist policy. Language models escaped that through in-context learning, and S1 attempts the same transition for robots.
Development environments are moving too. Google Antigravity 2.0 added Remote Control: connect from any web browser to Antigravity sessions running across laptops, servers, and desktops, retaining access to files, workspaces, build tools, and credentials. Beyond enabling it in the desktop app there is a headless daemon install path, with a one-time sign-in.
The documented reason ties back to everything above: as agents take on full-subsystem refactorings and extensive test runs, operations run long enough that a person cannot sit in front of them.
What to actually try this week
First, run an open-weight model yourself. Opening the zai-org/GLM-5.3-Flash or Qwen/Qwen3.8-Flash-Next model cards and reading the license and the runtime requirements already teaches a great deal. Both remain too large for most personal machines, so renting a cloud GPU by the hour is the realistic first step.
Second, count where your current tooling spends. Read one month of API billing through an inference lens; often it is agent loop iterations, not request counts, that create the bill.
Third, if you handle internal documents, examine whether that one workload can move to a local model. Not everything has to move. The point of Perplexity's design is a local default with approved, occasional trips outside.
Fourth, build the habit of conserving context: keep only the MCP servers you actually use and push the rest into skills that load on demand. This helps with frontier models as much as with local ones.
Taken together, chips, models, and runtimes point the same way this week. Where intelligence runs has become a choice again.