{"id":7970,"library":"azureml-inference-server-http","title":"Azure Machine Learning Inference Server (HTTP)","description":"The `azureml-inference-server-http` library provides the core HTTP server runtime for deploying machine learning models on Azure Machine Learning. It hosts user-defined model scripts (`score.py`) by loading `init()` and `run()` functions, allowing models to be exposed via a FastAPI-based REST API. As of version 1.5.1, it continues to evolve primarily through internal improvements and stability updates, with a focus on seamless integration into the Azure ML ecosystem.","status":"active","version":"1.5.1","language":"en","source_language":"en","source_url":"https://github.com/Azure/azureml-inference-server-http","tags":["azure","machine learning","inference","server","http","fastapi","runtime"],"install":[{"cmd":"pip install azureml-inference-server-http","lang":"bash","label":"Install core server components"}],"dependencies":[],"imports":[{"note":"This library provides the server runtime. User interaction is primarily by defining a `score.py` script with `init()` and `run()` functions, not by directly importing server components into application code.","wrong":"from azureml.inference.server.http import HttpServer","symbol":"HttpServer","correct":"Users define `init()` and `run()` functions in a `score.py` file."}],"quickstart":{"code":"# Create a file named 'score.py'\n# ---\n# import json\n# \n# def init():\n#     global model\n#     # In a real scenario, load your model here, e.g., from a file.\n#     model = {\"status\": \"initialized\"}\n# \n# def run(raw_data):\n#     try:\n#         data = json.loads(raw_data)\n#         prediction = f\"Model received input: {data.get('input', 'no input')} and is {model['status']}\"\n#         return json.dumps({\"output\": prediction})\n#     except Exception as e:\n#         return json.dumps({\"error\": str(e)})\n# ---\n# To run locally, save the above to 'score.py' and execute in your terminal:\n# export AZUREML_ENTRY_SCRIPT=score.py\n# python -m azureml.inference.server.http.http_server\n# \n# Then, send a request to http://localhost:5001/score\n# Example with curl:\n# curl -X POST -H \"Content-Type: application/json\" -d '{\"input\": \"example data\"}' http://localhost:5001/score\n","lang":"python","description":"This quickstart demonstrates how to create a minimal `score.py` file, which is the standard interface for models running on `azureml-inference-server-http`. It also provides the necessary shell commands to run this server locally for testing. The server exposes a `/score` endpoint for inference requests."},"warnings":[{"fix":"Focus on correctly structuring your `score.py` script and its environment, rather than attempting to import and use internal server components directly.","message":"This library is primarily a server runtime, not a client library for direct programmatic interaction with ML models. Users define `init()` and `run()` functions in a `score.py` file that this server loads and executes.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Before running `python -m azureml.inference.server.http.http_server`, ensure `export AZUREML_ENTRY_SCRIPT=your_score_file.py` (or `set AZUREML_ENTRY_SCRIPT=your_score_file.py` on Windows) is executed.","message":"Local testing requires setting specific environment variables, most notably `AZUREML_ENTRY_SCRIPT`, to point the server to your `score.py` file.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always refer to the official Azure ML documentation for the recommended `score.py` template and function signatures for your target Azure ML SDK and server version to ensure compatibility.","message":"The exact signature requirements for `init()` and `run()` in `score.py` have been stable, but future versions might introduce subtle changes or new optional parameters.","severity":"breaking","affected_versions":"Across major releases (e.g., 1.x to 2.x)"},{"fix":"When deploying to Azure ML, provide a `conda_env.yml` file. For local testing, manually `pip install` all necessary packages into your Python environment *before* running the server.","message":"All Python package dependencies for your model code within `score.py` must be explicitly managed and installed into the server's environment. `pip install azureml-inference-server-http` does not install your model's dependencies.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Ensure your `score.py` contains `def init():` at the module level. This function is called once when the server starts.","cause":"The `score.py` file does not contain a globally defined `init()` function, which is required by the inference server.","error":"Failed to find 'init' function in entry script."},{"fix":"Ensure your `score.py` contains `def run(raw_data):` at the module level. This function is called for each incoming inference request.","cause":"The `score.py` file does not contain a globally defined `run(raw_data)` function, which is required for processing inference requests.","error":"Failed to find 'run' function in entry script."},{"fix":"For local testing, `pip install your_custom_package` in your active environment. For Azure ML deployments, ensure 'your_custom_package' is listed in your `conda_env.yml` (or `requirements.txt`) file specified during deployment.","cause":"A Python package imported by your `score.py` (or its dependencies) is not present in the environment where the inference server is running.","error":"ModuleNotFoundError: No module named 'your_custom_package'"},{"fix":"Ensure your client sends a valid JSON string in the request body (e.g., `Content-Type: application/json`). Inside your `run` function, add robust error handling for `json.loads(raw_data)`.","cause":"The `run(raw_data)` function received input that is not valid JSON, or `raw_data` was not correctly parsed as a string.","error":"json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"}]}