Fluent model syntax
Deploy from your model codebase. Compose serving constraints in Python and choose performance terms on each call.
Implemented requests and deployment preview
The Python SDK now supports synchronous requests through infimal.Client.from_env().model(listing_id): chat, embeddings, speech, images and video. Stack, app.<model_type>, async request chains and bid() below remain design proposals. Use infimal.App for existing infrastructure declarations.
The serving classes and local assets in these examples belong to your project. Timings and budgets are illustrative, not measurements, accepted offers or published prices.
Requests available now
Install the current infimal SDK. Set INFIMAL_API_KEY and optionally INFIMAL_ENDPOINT; Client.from_env() reads them. Supply a listing id from GET /v1/models. A handle does not resolve Stack names or deploy anything. These examples are synchronous Python.
Each request submits once. Chat run() returns text and usage; chat stream() yields text deltas. Embeddings return vectors in input order. Speech run() returns bytes; its stream() yields transport chunks, not guaranteed audio frames. Image and video run() submit a durable job, poll it and return a JobResult with artifacts and bytes(). A polling timeout does not cancel the job; GlowError.details contains its job_id.
max_tokens() caps output tokens, not dollars. No request deadline, throughput target or cost bid is accepted: bid() raises NotImplementedError. LoRA selection and output shapes must be supported by the deployed listing and runtime.
from infimal import Client client = Client.from_env()chat = client.model("qwen3-8b")print(chat.chat("Explain this code.").max_tokens(128).run().text)for text in chat.chat("Summarize it.").max_tokens(64).stream(): print(text, end="", flush=True) vectors = client.model("your-embedding-listing").embeddings(["coast", "city"]).run()audio = client.model("your-speech-listing").speech("Hello.").speaker("alloy").run()image = client.model("your-image-listing").images("A paper boat.").preset("I1").seed(42).run()open("boat.png", "wb").write(image.bytes())Reference images without an inline body
create_upload() returns a tenant-scoped key, a signed PUT URL and required headers. PUT exactly the declared byte count and content type without your API key, then use reference_key(). PNG, JPEG and WebP are accepted. URLs expire after 15 minutes; input objects are pruned after 24 hours. The default limit is 32 MiB per upload, configurable with GLOW_MAX_UPLOAD_MIB up to 128 MiB.
The API validates the reference before reserving credit and reads it again before dispatch. Missing, oversized or malformed objects are refused; replacement after submission fails the job and refunds its hold. reference(path_or_bytes) remains available for inline base64 images. Uploaded inputs and generated artifacts have separate namespaces.
from pathlib import Pathimport httpxfrom infimal import Client client = Client.from_env()data = Path("reference.png").read_bytes()upload = client.create_upload("image/png", len(data))httpx.put(upload["url"], content=data, headers=upload["headers"]).raise_for_status()image = (client.model("your-image-listing").images("A product portrait.") .preset("I1").reference_key(upload["key"]).seed(42).run())Start in your model codebase
Keep infra.py alongside your model project. source="." resolves relative to that file, and entrypoint("serve:Chat") names the serving entrypoint in your codebase. Your project owns model loading, inference, local modules and dependency locks; IaC adds packaging, access and serving constraints.
Synthesis records an entrypoint reference without importing serving code or loading model weights. The builder imports it in the packaged environment. The entrypoint must implement a supported serving adapter contract; an arbitrary Python callable cannot automatically acquire disaggregation, adapter selection or streaming capabilities.
Runtime packaging follows the project package manifest and explicitly declared assets, with dependencies resolved from a lock. Infrastructure-only files, credentials and development caches are excluded. The packaged bytes and recipe determine the build digest. If runtime code depends on an excluded IaC file, packaging must fail or require restructuring; silently omitting a runtime dependency is not valid.
Editing only scaling or other deployment policies therefore leaves the artifact unchanged. Changes to included serving code, assets, initialization or dependency locks require a build. The CLI must report the exact packaged inputs before applying; a broad hash of every Python file in the repository cannot provide this separation.
my-model/
├── infra.py # declaration
├── serve.py # serving entrypoint / adapter
├── pyproject.toml # runtime package and dependencies
├── uv.lock # pinned dependencies
├── model/ # your implementation
└── weights/ # explicitly declared local assetsOne grammar, different model types
Stack collects a deployment declaration. Model-specific chains describe capabilities, serving constraints and scaling. They perform no remote work while being authored; the CLI synthesizes and applies the resulting specification.
Application code resolves a deployed model through Client().model("stack/name") in the selected environment. Calling that handle constructs a request. Output methods and bid() configure that request; run() or stream() submits it.
The call examples are async function bodies: import Client from infimal, place the body inside async def main(), and execute it using asyncio.run(main()). The voice example also expects your application's async audio player. Each definition is a standalone infra.py example; one stack can also contain several model declarations.
app.<model_type>(name, source=".").entrypoint(...).configure()
model(input).shape_output().bid(...).run()Chat: define
Bring your serving entrypoint. Separate prefill and decode, then choose the pace of each reply.
serve:Chat comes from your project. Its runtime must support prefill/decode separation.
from infimal import Stack, GiB app = Stack("studio") chat = ( app.llm("chat", source=".") .entrypoint("serve:Chat") .disaggregate() .kv_cache(min=24 * GiB) .scale(max=1000))Chat: call
First token, generation speed and a spending ceiling belong to this call.
from infimal import Client chat = Client().model("studio/chat") async for token in ( chat("Explain this code.") .bid(ttft="300ms", tokens_per_second=60, max_cost="$0.02", max_tokens=512) .stream()): print(token, end="", flush=True)Images: define
Deploy your image pipeline with its adapters. Compose every generation at call time.
Package your serving code and a compatible adapter; choose the adapter on each request.
from infimal import Stack app = Stack("studio") portraits = ( app.image("portraits", source=".") .entrypoint("serve:Portraits") .lora("editorial", "./weights/editorial.safetensors") .scale(max=100))Images: call
Choose the adapter, image size and seed, then submit the terms of this generation.
from infimal import Client portraits = Client().model("studio/portraits") image = await ( portraits("A portrait in soft afternoon light.") .with_lora("editorial", strength=0.8) .size(1024, 1024) .seed(42) .bid(finish_in="10s", max_cost="$0.05") .run())Video: define
Deploy from your video codebase. Give each shot an output shape, a deadline and a budget.
The declaration restricts outputs to resolutions and durations your runtime supports.
from infimal import Stack app = Stack("studio") film = ( app.video("film", source=".") .entrypoint("serve:Film") .resolutions("720p", "1080p") .max_duration("10s") .scale(max=32))Video: call
The completion deadline includes queueing. The cost ceiling covers this video.
from infimal import Client film = Client().model("studio/film") clip = await ( film("A paper boat in a neon-lit city.") .resolution("720p") .duration("5s") .bid(finish_in="5m", max_cost="$1.00") .run())Voice: define
Keep your speech model and loading code. Stream audio with a target on every call.
Expose the voices supported by your entrypoint. Latency is chosen on the request.
from infimal import Stack app = Stack("studio") voice = ( app.speech("narrator", source=".") .entrypoint("serve:Narrator") .voices("warm", "bright") .scale(max=200))Voice: call
Select a speaker and pace. Bid on first audio, then consume the audio stream.
from infimal import Client voice = Client().model("studio/narrator") async for audio in ( voice("Your next adventure starts here.") .speaker("warm") .speed(1.1) .bid(first_audio="200ms", max_cost="$0.01") .stream()): await player.write(audio)Embeddings: define
Package your encoder, normalize its vectors and give background work room to wait.
Normalize output vectors and permit batches of up to 128 compatible inputs.
from infimal import Stack app = Stack("studio") search = ( app.embedder("search", source=".") .entrypoint("serve:Embedder") .normalize() .batch(max=128))Embeddings: call
This submission gets its own completion deadline and total spending ceiling.
from infimal import Client search = Client().model("studio/search")documents = ["The quiet coast.", "A city after rain."] vectors = await ( search(documents) .bid(finish_in="1h", max_cost="$0.50") .run())Registry models are another source
When the platform provides a compatible managed runtime, a registry identifier is a shorthand for that recipe. Planning resolves and locks an immutable model revision. This remains an alternative to deploying your own codebase; it does not bypass model/runtime compatibility checks.
The identifier below illustrates source syntax. It is not a claim that this model or managed deployment path is currently offered by Infimal.
chat = app.llm("chat", "Qwen/Qwen3-32B").disaggregate()Deployment constraints
| Method | Proposed contract |
|---|---|
| llm / image / video / speech / embedder | Create a named model service in the same deployment graph. Model type selects the serving contract, not an automatic implementation of missing capabilities. |
| entrypoint("module:symbol") | Reference the project's serving entrypoint without importing it during synthesis. Validate its adapter contract in the build. |
| disaggregate() | Require separate prefill and decode pools with disjoint device allocations. They may share a host. Omission permits automatic selection; unsupported separation is an error. |
| kv_cache(min=24 * GiB) | Require at least 24 GiB of usable device KV capacity per ready decode replica across its shards, excluding weights and activations. Without disaggregation the floor applies to each ready unified replica. It is capacity, not occupied bytes. |
| scale(max=1000) | Set a replica ceiling; it does not reserve or guarantee 1,000 available replicas. Zero running replicas do not reserve the KV floor. With disaggregation this cap counts complete prefill and decode replicas in total. |
| lora(name, path) | Declare an available adapter. Validate compatibility with the base model and pin the adapter content. |
| resolutions / max_duration / voices | Restrict supported outputs. Declaring a setting does not give the underlying model a capability it lacks. |
| normalize / batch | Specify output normalization and the maximum compatible execution batch. The engine forms batches within accepted request terms. |
Request terms and the fee market
TTFT, generation speed, first-audio targets, completion deadlines and fee ceilings belong to the call. They do not change the deployment. Output shape, speaker, seed and adapter selection also belong to the request.
The public bid expresses bounds and willingness to pay. Customers do not bid on the internal per-resource fee vector. With no overrides, requests use the model's published standard terms. Admission chooses an eligible offer; accepted rates and service terms remain fixed for the request even if internal congestion fees change.
If no eligible offer meets the bounds and ceiling, refuse clearly. Queue only while the request's time bounds permit. Never silently exceed the cost ceiling or relax a hard requirement. A requested bound is not an accepted guarantee until admission succeeds.
| Argument | Proposed meaning |
|---|---|
| ttft | Maximum requested time from API ingress to the first output token, including queueing and internal handoff. |
| tokens_per_second | Minimum requested per-request decode rate under the accepted metric definition. It is not aggregate fleet throughput. |
| first_audio | Maximum requested time from API ingress to the first playable audio chunk. |
| finish_in | Completion deadline relative to API ingress, including queueing. |
| max_cost | Total USD ceiling for this submitted call or finite stream, covering billable input, cached input, output or media units under the accepted rates. It is not a per-million-token rate. |
| max_tokens | Output-token cap used to bound generation and admission cost; ending early is billed under the accepted usage terms. |
Execution and validation
- run() is awaited for one result; stream() returns an async iterator for model types that support incremental output.
- Each request expression is one submission. Reuse after submission is rejected; retry requires an explicit operation/idempotency contract.
- Constructors and modifiers validate names, units and allowed values. Unknown settings and unsupported combinations fail rather than being dropped.
- Model recipes, adapters and runtime compatibility are checked before applying. Request shapes and offered terms are checked again at admission.
- This preview does not replace the current custom-code snapshot/setup lifecycle. The serving adapter contract and build path must integrate that lifecycle before this API ships.