MLflow CrewAI autologging records LLM provider API keys in logged trace spans
Summary
mlflow.crewai.autolog() leaks CrewAI LLM provider API keys into MLflow trace telemetry.
When a CrewAI Agent is instrumented with MLflow autologging, MLflow serializes the agent's __dict__ and converts its fields to strings. The llm field contains the CrewAI LLM object, whose string representation includes its api_key in plaintext.
As a result, an application that calls mlflow.crewai.autolog() can unintentionally send its LLM provider API key to the MLflow trace backend or configured OTLP destination.
The application does not need to explicitly log the credential or place it in a span attribute. The credential is extracted automatically by MLflow's instrumentation.
This was verified on MLflow 3.16.0 with CrewAI 1.15.18 using MLflow's own OTLP exporter. The fake credential appeared in 5 attribute locations across 2 spans.
Details
The primary issue is in:
mlflow/crewai/autolog.py::_get_agent_attributes()
The function walks the CrewAI Agent.__dict__ and converts agent fields to strings. tools and embedder receive special handling, but credential-bearing objects such as the following are not sanitized:
llmfunction_calling_llmagent_executorcrew
For example, agent.llm is a CrewAI LLM object. In CrewAI 1.15.18, the underlying LLM model contains api_key as a normal field. Its string representation...
Affected versions
| Package | Affected | Fixed in |
|---|---|---|
| mlflow PyPI | >= 2.19.0, < 3.17.0 | 3.17.0 |
Details and references
### Summary `mlflow.crewai.autolog()` leaks CrewAI LLM provider API keys into MLflow trace telemetry. When a CrewAI `Agent` is instrumented with MLflow autologging, MLflow serializes the agent's `__dict__` and converts its fields to strings. The `llm` field contains the CrewAI LLM object, whose string representation includes its `api_key` in plaintext. As a result, an application that calls `mlflow.crewai.autolog()` can unintentionally send its LLM provider API key to the MLflow trace backend or configured OTLP destination. The application does not need to explicitly log the credential or place it in a span attribute. The credential is extracted automatically by MLflow's instrumentation. This was verified on **MLflow 3.16.0** with **CrewAI 1.15.18** using MLflow's own OTLP exporter. The fake credential appeared in **5 attribute locations across 2 spans**. ### Details The primary issue is in: `mlflow/crewai/autolog.py::_get_agent_attributes()` The function walks the CrewAI `Agent.__dict__` and converts agent fields to strings. `tools` and `embedder` receive special handling, but credential-bearing objects such as the following are not sanitized: * `llm` * `function_calling_llm` * `agent_executor` * `crew` For example, `agent.llm` is a CrewAI `LLM` object. In CrewAI 1.15.18, the underlying LLM model contains `api_key` as a normal field. Its string representation therefore contains the credential: ```text llm_type='openai' model='gpt-4.1-mini' ... api_key='sk-proj-...' ... ``` MLflow subsequently stores this string as a span attribute. There is an important indication that this is an unintended disclosure: MLflow's dedicated LLM attribute extraction already explicitly excludes `api_key` in `_get_llm_attributes()`. The LLM span therefore does not expose the credential, while the broader Agent `__dict__` serialization bypasses that protection. There is also a second, independent serialization path in: `mlflow/crewai/autolog.py::_construct_full_inputs()` This code uses: ```python v.__dict__ if hasattr(v, "__dict__") else v ``` on call arguments. During CrewAI execution, the `Agent` object is passed as an argument, causing its `__dict__` to be expanded and the credential-bearing LLM object to be included in `mlflow.spanInputs`. Therefore, there are two independent leakage paths: 1. **Agent span attributes** through `_get_agent_attributes()` 2. **Span inputs** through `_construct_full_inputs()` Fixing only the Agent attribute serialization would leave the `mlflow.spanInputs` path vulnerable. A safer approach would be to avoid recursively serializing arbitrary object `__dict__` values. Agent telemetry should use an explicit allowlist of non-sensitive scalar fields, and credential-bearing objects such as `llm`, `function_calling_llm`, `agent_executor`, and `crew` should either be reduced to safe metadata or omitted. Similarly, `_construct_full_inputs()` should not automatically expand arbitrary objects through `__dict__`, or the resulting values should pass through the same sanitization mechanism before being attached to a span. ### PoC Environment: ```text MLflow: 3.16.0 CrewAI: 1.15.18 Python: 3.12.3 opentelemetry-exporter-otlp-proto-http: 1.42.1 ``` #### 1. Direct serialization proof The following demonstrates the vulnerability without starting a Crew, making a network request, or requiring an MLflow server: ```python import importlib from crewai import LLM, Agent autolog = importlib.import_module("mlflow.crewai.autolog") llm = LLM( model="gpt-4.1-mini", api_key="sk-proj-EXAMPLE1234567890", ) agent = Agent( role="r", goal="g", backstory="b", llm=llm, ) attrs = autolog._get_agent_attributes(agent) assert "sk-proj-EXAMPLE1234567890" in attrs["llm"] print(attrs["llm"]) ``` The output contains the API key in plaintext, for example: ```text llm_type='openai' model='gpt-4.1-mini' ... api_key='sk-proj-EXAMPLE1234567890' ... ``` No provider request is required. #### 2. End-to-en