Serverless Inference in plain English.

Serverless inference means deploying a model as an endpoint that the platform starts when a request arrives and shuts down when traffic stops. You are billed for compute seconds, not for a machine sitting idle. It is the model-hosting equivalent of AWS Lambda, and providers like RunPod, Modal, Baseten, and Replicate all offer a version of it. The appeal is obvious for bursty or low-traffic apps: a side project that gets ten requests a day costs cents rather than hundreds of dollars a month for a dedicated GPU. The catch is the cold start: loading a multi-gigabyte model onto a fresh GPU can take anywhere from a few seconds to a minute, so the first request after a quiet period is slow. Platforms fight this with warm pools, snapshotting, and minimum-instance settings, each of which costs money and erodes the pay-per-use advantage. Serverless inference suits variable traffic and experiments; steady high traffic is usually cheaper on dedicated capacity.