Skip to main content

InferenceEngineDetail

Inference Engine Object

id string

The unique identifier for the inference engine

Example: engine-12345
name string
Example: test-inference-engine
status string

Possible values: [Creating, Active, Inactive]

Example: Active
base_model string
Example: Llama-3.2-3B-Instruct
task_type string
Example: text-generation
server_type string
Example: on-demand
deployments object[]
  • Array [
    • id string

    The unique identifier for the deployment

    Example: dp-12345
    name string

    The name of the deployment

    Example: deployment-1
    status string

    The current status of the deployment

    Possible values: [Pending, Deploying, Deployed, Failed, Cancelled, Terminated, Deactivated, Reactivating, TimedOut]

    Example: Deployed
    created_at integer

    The timestamp when the deployment was created

    Example: 1633036800
  • ]
    • base_model_option string
    Example: pre-trained
    last_access_timestamp integer
    Example: 1633036800
    resource_management_config object
    oneOf
    management_type string
    Example: inactive_timeout
    management_config object
    inactive_duration integer

    Period of inactivity

    Example: 3600
    created_at integer

    The timestamp when the deployment was last updated

    Example: 1633036800
    InferenceEngineDetail
    {
      "id": "engine-12345",
      "name": "test-inference-engine",
      "status": "Active",
      "base_model": "Llama-3.2-3B-Instruct",
      "task_type": "text-generation",
      "server_type": "on-demand",
      "deployments": [
        {
          "id": "dp-12345",
          "name": "deployment-1",
          "status": "Deployed",
          "created_at": 1633036800
        }
      ],
      "base_model_option": "pre-trained",
      "last_access_timestamp": 1633036800,
      "resource_management_config": {
        "management_type": "inactive_timeout",
        "management_config": {
          "inactive_duration": 3600
        }
      },
      "created_at": 1633036800
    }