Build for unpredictable demand.
Infimal turns infrastructure as code into model services that can scale down between requests and out when demand arrives.
A complete service, from source to snapshot.
02 / REFERENCEFind your command ↗Infrastructure and operations from your terminal.
Your model is the unit of deployment
Bring the model and the code that runs it. Declare its access policy, replica bounds, and batching behavior. Infimal translates that declaration into the specification it uses to build, place, and operate the service.
Python is the current authoring interface. The deployment specification is the infrastructure contract. GPU selection, placement, and routing belong to the platform.
Three stages, one definition
| Stage | You define | Infimal handles |
|---|---|---|
| Declare | App, Endpoint, Scale, Batch | Validate and serialize the deployment specification |
| Initialize | @app.setup | Capture initialized process and model state |
| Serve | @app.handler or @app.stream | Restore, place, and route work |
Designed for both ends of the curve
A new model may get no requests for hours and a burst minutes later. A fixed fleet forces you to decide how much capacity to hold before you know that demand. Infimal is designed around retaining reusable model state and allocating compute as work arrives.
Set the bounds your service may operate within. A maximum of 1,000 replicas is a configuration limit, not a reservation or a guarantee that 1,000 replicas are immediately available.
Your first service
Follow the quickstart to create a private service, inspect its plan, and understand how it is deployed. Use the CLI reference for exact commands and the infrastructure reference for declaration fields.
python -m infimal plan asr-emotion-checkpoint4200
python -m infimal deploy asr-emotion-checkpoint4200
infimal apps status asr-emotion-checkpoint4200