Pistachio v0.26: Canary Rewrite, Error Classification, and Inference Pipeline Refactor

Pistachio v0.26. Inference is now claim-based: clients claim a lane, get exclusive access to a session, and send requests one at a time. The TypeScript SDK, the canary test, and the load test all use this flow. Error classification shows specific failure categories instead of generic messages. Ten critical bugs fixed.

Canary test rewrite

The canary test no longer uses low-level TCP probing. It uses the same claim-based inference path as production: claim a lane, send configurable sequential prompts (default 5), release. This tests the real end-to-end flow rather than just checking whether a TCP connection succeeds.

Canary errors now show specific categories instead of "unknown". Failure types include provider_overloaded, ecies_encrypted, provider_rate_limited, tcp_timeout, model_not_found, and others. Each category maps to a specific failure mode in the Morpheus protocol, which makes diagnosis faster.

Session-aware SDK (v0.3.0)

Morpheus providers handle one prompt at a time per session. If two requests hit the same session concurrently, the provider's proxy-router queues or drops the second one. Previous versions of Pistachio did not track this. Multiple requests could pile up on the same session, and the result was unpredictable: timeouts, duplicate responses, or silent failures.

v0.26 enforces the constraint at two levels. Server-side, an InFlightTracker holds a set of active session IDs. A second request on a busy session gets an immediate 429 instead of being forwarded to the provider. Client-side, the TypeScript SDK mirrors this with an in-flight flag per lane. The SDK will not send a second request on a lane that already has one in progress.

The claim system ties it together. `client.claim("kimi-k2.6")` returns a `PistachioLane` object that owns exclusive access to one session. Call `lane.chat(messages)` to send a prompt. The lane tracks request count, error count, and expiry. Call `lane.release()` when done. If the lane is dropped without release, a server-side guard cleans it up.

The SDK is optional. The OpenAI and Anthropic-compatible APIs still work without it. Send a request without a session ID and Pistachio picks an available session automatically via round-robin. The SDK adds finer-grained control: you choose which session to use, you own the lane exclusively, and you manage the lifecycle yourself.

const client = new PistachioClient({ baseUrl, apiKey });
const lane = await client.claim("kimi-k2.6");
const res = await lane.chat([{ role: "user", content: "Hello" }]);
await lane.release();

Top Models stock template

A new "Top Models" plan template ships with v0.26. It includes kimi-k2.5, glm-5, glm-5.1-non-thinking, and kimi-k2.6, each configured at 10 MOR with 10 sessions (400 MOR total). This provides a working starting configuration for users who want to test current high-performing models without manually configuring bids.

Load test tool: parallel model execution

The built-in load test now runs all models in parallel instead of testing them one at a time. The previous sequential approach caused later models' sessions to expire before the test reached them. Parallel execution keeps all sessions active and completes in the duration of one model window instead of N.

8 critical bugs fixed

Tokio deadlock: async lock held across an await point caused the runtime to hang under load. Replaced with scoped lock acquisition.

Lane gating: lanes could be assigned to requests even when the session was not ready. Gating now checks session state before assignment.

in_use flag reset: background refresh was resetting in_use flags on active sessions, breaking concurrent distribution. Merge logic now preserves in_use state.

Global init mutex: initialization held a global lock that blocked all other operations. Replaced with per-resource locks.

Retry self-DDoS: failed requests retried immediately with no backoff, amplifying load on struggling providers. Removed retry loop, single attempt, fail fast.

TX send timeout: on-chain transactions could hang indefinitely. 30-second timeout now applies to all transaction sends.

Forwarding internal fields: internal metadata fields were being forwarded to providers in request payloads. Stripped before forwarding.

Remaining sync locks: several std::sync locks in async code paths replaced with tokio::sync alternatives.

Install or upgrade

Desktop app users will receive an auto-update prompt on next launch. Full documentation at drm3.io/products/pistachio. Release notes and binaries at github.com/drm3labs/drm3-releases. Docker image at hub.docker.com/r/drm3/pistachio.

docker pull drm3/pistachio:latest

brew tap drm3labs/drm3 && brew install --cask pistachio-desktop

brew tap drm3labs/drm3 && brew install pistachio

curl -fsSL https://drm3.network/install.sh | sh

Disclosure: DRM3 Labs holds and stakes MOR and sells MOR-denominated products.