Multi Model Server (Deprecated)

1.1.11 · deprecated · verified Thu Apr 16

Multi Model Server (MMS) was an AWS Labs tool designed for serving neural network models for inference. This specific PyPI package and its underlying project have been deprecated and are no longer actively maintained. Users are strongly encouraged to migrate to TorchServe for ongoing support and new features. The last released version was 1.1.11.

Common errors

```
ModuleNotFoundError: No module named 'multi_model_server'
```
cause Attempting to `import multi_model_server` as a typical Python library, which is not its primary design or use case.

fix The `multi-model-server` package is primarily a CLI tool. After installation, use `multi-model-server --start` from your terminal. If you need a programmatic Python API for model serving, consider TorchServe's client libraries or APIs.
```
Using multi-model-server, but need a feature available in TorchServe (e.g., advanced metrics, specific model formats, newer runtime environment).
```
cause The `multi-model-server` project is deprecated and does not receive new features, bug fixes, or compatibility updates.

fix The only long-term solution is to migrate your model serving workloads from `multi-model-server` to TorchServe, which is the actively developed successor and offers continued feature enhancements.
```
CVE-2021-44228: Apache Log4j2 JNDI Remote Code Execution vulnerability
```
cause Older versions of `multi-model-server` bundled vulnerable versions of the Log4j library. While some patches were released for this specific package, the overall project is deprecated, meaning future vulnerabilities might not be addressed.

fix Upgrade to `multi-model-server` version `1.1.8` or newer if you absolutely must use this deprecated package. However, the definitive and recommended fix is to migrate to TorchServe, which actively maintains up-to-date dependencies and addresses security concerns.

Warnings

breaking The `multi-model-server` project is officially deprecated and is no longer actively maintained. All active development has shifted to TorchServe.
Fix: Migrate your model serving infrastructure to TorchServe (https://pytorch.org/serve/) for continued support, security updates, and new features.
gotcha This package will not receive security updates or bug fixes. Continuing to use it in production environments poses significant security risks due to unpatched vulnerabilities.
Fix: Immediately plan and execute migration to TorchServe to ensure your serving environment is secure and supported.
gotcha The `multi-model-server` PyPI package primarily provides a command-line interface (CLI) tool. It does not offer a standard Python library API for programmatic `import` and direct interaction.
Fix: Interact with the server via its command-line interface or HTTP API. If you require programmatic Python control over a model server, TorchServe provides more appropriate client libraries and APIs.

Install

pip install multi-model-server Install Deprecated Multi Model Server

Imports

Multi Model Server CLI
```
multi-model-server --start
```
The `multi-model-server` package primarily provides a command-line interface (CLI) for starting the server, rather than a Python programmatic API for direct imports. No common Python library symbols are exposed for direct import and usage.

Quickstart

This quickstart demonstrates how to install and start the deprecated Multi Model Server using its command-line interface. Note that for actual model serving, you would typically archive models first. Given the project's deprecated status, this is provided for historical context or existing deployments only.

# 1. Install the deprecated server
pip install multi-model-server

# 2. (Optional, but highly recommended for actual serving) Create a model archive.
#    Note: torch-model-archiver is now part of TorchServe installation. 
#    If you need to archive models for this deprecated MMS, you might need
#    an older, compatible version of torch-model-archiver or manually bundle.
#    Example using (potentially incompatible) current torch-model-archiver syntax:
#    torch-model-archiver --model-name mymodel --version 1.0 --model-file model.py --handler handler.py

# 3. Start the Multi Model Server
multi-model-server --start

view raw JSON →