{"id":9876,"library":"kserve","title":"KServe Python SDK","description":"The KServe Python SDK provides a client library for interacting with KServe (formerly KFServing) on Kubernetes. It allows users to define, deploy, and manage machine learning inference services programmatically. The current version is 0.17.0. Releases are typically aligned with the main KServe project, with new versions dropping every few months.","status":"active","version":"0.17.0","language":"en","source_language":"en","source_url":"https://github.com/kserve/kserve","tags":["machine learning","kubernetes","serving","inference","model deployment","mle","mlops"],"install":[{"cmd":"pip install kserve kubernetes","lang":"bash","label":"Install KServe SDK and Kubernetes client"}],"dependencies":[{"reason":"Required to interact with Kubernetes clusters and manage KServe resources.","package":"kubernetes","optional":false}],"imports":[{"symbol":"KServeClient","correct":"from kserve import KServeClient"},{"note":"Class moved from deprecated kfserving package to kserve package and structure.","wrong":"from kfserving.models import V1beta1InferenceService","symbol":"V1beta1InferenceService","correct":"from kserve import V1beta1InferenceService"},{"symbol":"constants","correct":"from kserve import constants"},{"note":"The Kubernetes client is a direct dependency for cluster interaction.","symbol":"client","correct":"from kubernetes import client as k8s_client"}],"quickstart":{"code":"import os\nfrom kubernetes import client as k8s_client\nfrom kserve import KServeClient, constants, utils\nfrom kserve import V1beta1InferenceService, V1beta1InferenceServiceSpec, V1beta1PredictorSpec, V1beta1SKLearnSpec\n\n# --- Configuration and Client Initialization ---\n# This example assumes kubectl is configured to connect to a Kubernetes cluster.\n# For in-cluster execution, uncomment `k8s_client.config.load_incluster_config()`.\n# For local execution, ensure your ~/.kube/config is set up.\ntry:\n    k8s_client.config.load_kube_config()\nexcept k8s_client.config.config_exception.ConfigException:\n    print(\"Warning: Could not load kube-config. Attempting in-cluster config.\")\n    try:\n        k8s_client.config.load_incluster_config()\n    except k8s_client.config.config_exception.ConfigException:\n        print(\"Error: Could not load any Kubernetes config. Please ensure kubectl is configured or run within a cluster.\")\n        exit(1)\n\napi_version = constants.KSERVE_API_VERSION\nkserve_client = KServeClient()\n\nnamespace = os.environ.get('K8S_NAMESPACE', 'default') # Use an environment variable or default\nservice_name = 'sklearn-iris-quickstart'\n\n# --- Define an InferenceService ---\nisvc = V1beta1InferenceService(\n    api_version=api_version,\n    kind=constants.KSERVE_KIND,\n    metadata=k8s_client.V1ObjectMeta(\n        name=service_name, namespace=namespace\n    ),\n    spec=V1beta1InferenceServiceSpec(\n        predictor=V1beta1PredictorSpec(\n            sklearn=V1beta1SKLearnSpec(\n                storage_uri='gs://kfserving-examples/models/sklearn/iris',\n                protocol_version='v1'\n            )\n        )\n    )\n)\n\nprint(f\"Creating InferenceService '{service_name}' in namespace '{namespace}'...\")\n# --- Create and Wait for InferenceService ---\ntry:\n    kserve_client.create(isvc)\n    print(f\"InferenceService '{service_name}' created. Waiting for it to be ready...\")\n    kserve_client.wait_isvc_ready(service_name, namespace=namespace)\n    print(f\"InferenceService '{service_name}' is ready:\")\n    print(kserve_client.get(service_name, namespace=namespace))\n    # Example of how to delete the service:\n    # kserve_client.delete(service_name, namespace=namespace)\n    # print(f\"InferenceService '{service_name}' deleted.\")\nexcept Exception as e:\n    print(f\"Failed to create or wait for InferenceService: {e}\")\n    # Attempt cleanup if creation partially succeeded but failed later\n    try:\n        kserve_client.delete(service_name, namespace=namespace)\n        print(f\"Attempted cleanup of '{service_name}'.\")\n    except Exception as cleanup_e:\n        print(f\"Failed to clean up '{service_name}': {cleanup_e}\")\n","lang":"python","description":"This quickstart demonstrates how to initialize the KServe client, define an InferenceService for a scikit-learn model, and deploy it to a Kubernetes cluster. It requires the `kubernetes` package and a configured `kubectl` context or running inside a Kubernetes pod."},"warnings":[{"fix":"Uninstall `kfserving` and `pip install kserve`. Update all imports from `from kfserving import ...` to `from kserve import ...`.","message":"The project was renamed from KFServing to KServe. The Python package `kfserving` is deprecated and no longer maintained. Users must migrate to the `kserve` package.","severity":"breaking","affected_versions":"<=0.7.0 (kfserving) to >=0.8.0 (kserve)"},{"fix":"Ensure you install both: `pip install kserve kubernetes`. You will also need a correctly configured Kubernetes context (e.g., `~/.kube/config`) or run within a cluster.","message":"The KServe Python SDK relies heavily on the official `kubernetes` Python client to interact with your cluster. This dependency is not automatically installed with `pip install kserve`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always refer to the official KServe documentation for the specific version of KServe you are running on your cluster. Use the `constants.KSERVE_API_VERSION` provided by the SDK to ensure compatibility where possible, but be aware of potential manifest differences.","message":"KServe's underlying Kubernetes API objects (like InferenceService) are evolving. While `v1beta1` is still widely used and supported by the SDK, future versions of KServe may promote `v1` as the primary API.","severity":"gotcha","affected_versions":"All versions, particularly relevant for KServe >= 0.10.0"}],"env_vars":null,"last_verified":"2026-04-17T00:00:00.000Z","next_check":"2026-07-16T00:00:00.000Z","problems":[{"fix":"Uninstall the old `kfserving` package and install the new `kserve` package: `pip uninstall kfserving; pip install kserve`. Then update your import statements to `from kserve import ...`.","cause":"You are trying to import from the old, deprecated 'kfserving' package.","error":"ModuleNotFoundError: No module named 'kfserving'"},{"fix":"Ensure your `kubectl` is configured and can connect to your cluster. If running inside a cluster, use `k8s_client.config.load_incluster_config()` (often handled by the SDK implicitly or in the quickstart example).","cause":"The Kubernetes client cannot find or access a valid Kubernetes configuration file (e.g., `~/.kube/config`) to connect to a cluster.","error":"kubernetes.config.config_exception.ConfigException: Cannot load kube-config from any of: [...]"},{"fix":"Check the latest KServe Python SDK documentation for the correct method names. For example, `create_isvc` was replaced by a more generic `create(isvc_obj)` method.","cause":"You are using an older API method that has been replaced or renamed in newer KServe SDK versions.","error":"AttributeError: 'KServeClient' object has no attribute 'create_isvc'"}]}