Skip to main content

Costs and FAQ

What does it cost to run?​

  • Serverless: $0.05 per million tokens.
  • Dedicated (enterprise only): $3/hr for models under 30B parameters. Contact us for larger models. Bigger models are slower, so the effective cost on a 27B model is roughly $0.15 per million tokens, depending on your workload pattern. Steady, high-volume traffic that keeps the GPU busy lands at the low end.

Trained models are often cheaper to run than zero-shot ones. A zero-shot request carries every class description, while a trained model learns what the classes mean from your examples, so requests are shorter.

What does it cost to train?​

Training a classifier or regressor is cheap, and you can start small.

  • LoRA fine-tune, typical: for 3-5 classes with about 100 samples each, training usually costs under $15.
  • LoRA fine-tune, larger datasets: training on up to about 5,000 samples costs under $50 at most.

Start with the smaller dataset to confirm your labels and setup work, then scale up to more data if your held-out score is still improving.

How can this be deployed?​

  • Serverless on Qwen3-8B. Pay per token with no engine to manage.
  • Dedicated for everything else (enterprise only). Any other backbone runs on a dedicated engine, which is available to enterprise customers only.

Use dedicated when you have high throughput or a predefined time window, such as scheduled workflows. Use serverless when your workload is spiky, low-volume, or unpredictable. Autoscaling is also available on dedicated engines, so reach out if you want it set up for your workload.

See Inference concepts, Deployments, and Autoscaling.

Which base model should I pick?​

Choose Qwen3-8B (not Qwen3-8B-Base) because it is the one you can deploy serverless.

  • Go bigger if you need a smarter model, for example when the task is subtle and the smaller model's held-out score is still improving.
  • Go smaller if your eval is already maxed out. A simpler task doesn't need a bigger model, and a smaller one is cheaper and faster.

Bigger and smaller backbones run on dedicated engines, which are enterprise only. If you're not on an enterprise plan, use Qwen3-8B.

Classification vs. regression: how do I choose?​

Choose by the shape of your answer.

  • Use classification when the answer is a choice among known options, such as route A or B, safe or unsafe, or which of five intents. You get a probability per class.
  • Use regression when the answer is a point on a scale, such as urgency from 0 to 1, a quality score, or a sentiment intensity. You get a single number.

Both train and deploy the same way. The only difference is the loss function used in training, so a regressor is served from a classification inference engine just like a classifier.

If your labels are really ordered buckets (low, medium, high) and you have enough examples at a finer grain, regression often uses your data more efficiently. If you only ever act on a few discrete outcomes, classification is simpler to threshold and evaluate.