{"library":"mlserver","title":"MLServer","description":"MLServer is an open-source inference server for machine learning models, designed to serve any ML framework through a standard V2 inference protocol. It aims to provide a lightweight and performant solution for deploying models and supports both REST and gRPC endpoints. The current version is 1.7.1, and it is actively developed and maintained by SeldonIO with a regular release cadence.","language":"python","status":"active","last_verified":"Sun May 17","install":{"commands":["pip install mlserver","pip install mlserver[all]"],"cli":{"name":"mlserver","version":"mlserver, version 1.7.1"}},"imports":["from mlserver import MLModel","from mlserver.types import InferenceRequest","from mlserver.types import InferenceResponse","from mlserver.cli import main"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"from mlserver import MLModel\nfrom mlserver.types import InferenceRequest, InferenceResponse, ResponseOutput\nimport numpy as np\n\nclass MyModel(MLModel):\n    async def load(self):\n        # In a real scenario, load your model artifacts here\n        self.model = lambda x: x * 2 # A simple dummy function\n        self.ready = True\n\n    async def predict(self, request: InferenceRequest) -> InferenceResponse:\n        input_data = request.inputs[0].data.__root__\n        input_array = np.array(input_data).astype(np.float32)\n        output_array = self.model(input_array)\n        \n        return InferenceResponse(\n            outputs=[\n                ResponseOutput(\n                    name=\"output-0\",\n                    shape=output_array.shape,\n                    datatype=\"FP32\",\n                    data=output_array.tolist(),\n                )\n            ]\n        )\n\n# To run this model:\n# 1. Save the above code as `model.py` in an empty directory.\n# 2. Open your terminal in that directory.\n# 3. Ensure mlserver and numpy are installed: `pip install mlserver numpy`\n# 4. Run the MLServer: `mlserver start .`\n#\n# You can then send an inference request (e.g., using curl in a new terminal):\n# curl -X POST 'http://localhost:8080/v2/models/MyModel/infer' \\\n#      -H 'Content-Type: application/json' \\\n#      -d '{\n#            \"inputs\": [\n#              {\n#                \"name\": \"input-0\",\n#                \"shape\": [1, 2],\n#                \"datatype\": \"FP32\",\n#                \"data\": [10.0, 20.0]\n#              }\n#            ]\n#          }'","lang":"python","description":"This quickstart defines a simple `MyModel` that doubles its input. Save the code as `model.py`, then use the `mlserver start .` command to run the server. An example `curl` command is provided to demonstrate how to send an inference request to the running server.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-17","installed_version":"1.7.1","pypi_latest":"1.7.1","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":40,"avg_install_s":18.6,"avg_import_s":2.98,"wheel_type":"wheel"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"mlserver","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"all","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"mlserver","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":22.6,"import_time_s":2.61,"mem_mb":57.5,"disk_size":"259M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"all","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":22.4,"import_time_s":2.51,"mem_mb":57.5,"disk_size":"259M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"mlserver","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"all","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"mlserver","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":19.8,"import_time_s":3.59,"mem_mb":63.5,"disk_size":"279M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"all","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":19.3,"import_time_s":3.46,"mem_mb":63.5,"disk_size":"279M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"mlserver","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"all","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"mlserver","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":17.3,"import_time_s":3.67,"mem_mb":62.7,"disk_size":"274M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"all","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":16.9,"import_time_s":3.61,"mem_mb":62.7,"disk_size":"274M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"mlserver","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"all","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"mlserver","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":15.3,"import_time_s":2.18,"mem_mb":51.3,"disk_size":"261M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"all","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":15,"import_time_s":2.22,"mem_mb":51.3,"disk_size":"261M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"mlserver","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"all","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"mlserver","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":25.6,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"all","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":25.3,"import_time_s":null,"mem_mb":null,"disk_size":null}]}}