feat(providers): add ProviderCapabilities dataclass with unified capability detection

[tcconnally]

Summary

Adds a structured ProviderCapabilities dataclass that aggregates previously scattered capability checks into a single object, addressing #6220.

Problem

CrewAI currently checks provider capabilities through separate methods (supports_response_schema(), supports_function_calling(), supports_stop_words()) and hardcoded provider lists. This means:

• Error messages are inconsistent (some mention "model", others "provider")
• There is no single source of truth for what a provider supports
• Adding new capability validation requires adding new scattered methods

Solution

ProviderCapabilities dataclass

@dataclass
class ProviderCapabilities:
    supports_response_format: bool = True
    supports_tool_calling: bool = True
    supports_reasoning: bool = False
    supports_streaming: bool = True
    supports_image_input: bool = False
    supports_stop_words: bool = True

LLM.get_capabilities()

Uses litellm introspection when available, falling back to static provider allowlists:

caps = llm.get_capabilities()
# caps.supports_response_format -> bool
# caps.supports_tool_calling -> bool
# caps.supports_reasoning -> bool
# etc.

Updated validation

_validate_call_params() now uses capabilities for both response_format and reasoning_effort validation with clearer error messages:

"The provider openrouter does not support response_format. Remove response_format, use result_as_string=True, or switch to a provider that supports structured output."

Static fallback allowlists

When litellm is unavailable, three frozensets provide the fallback:

• _PROVIDERS_WITHOUT_RESPONSE_FORMAT: deepseek, ollama, ollama_chat, hosted_vllm
• _PROVIDERS_WITHOUT_TOOL_CALLING: (empty — all modern providers support tools)
• _PROVIDERS_WITH_REASONING: openai, azure, anthropic, bedrock, cerebras

Related

• Closes feat: provider capability detection to prevent unsupported response_format / tool_call failures #6220
• Replaces the hardcoded deepseek check proposed in fix: skip response_format for Deepseek (provider doesn't support it) #6171 with a scalable approach
• Related to [BUG] OpenAI API call fails with "response_format type is unavailable now" when using Deepseek #5990 (DeepSeek response_format crash)

Summary by CodeRabbit

• Bug Fixes

  • Improved LLM parameter validation so unsupported options now fail fast with clearer, more specific error messages—especially for structured response formats and reasoning effort.
  • Added capability-aware checks for tool/function calling, multimodal/image input, stop-word handling, and streaming support.

• Chores

  • Centralized model capability detection into a single capability reporting mechanism to keep provider behavior consistent and easier to validate.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 81e9080c-01e8-43dc-bfb8-7a50c506518c

📥 Commits

Reviewing files that changed from the base of the PR and between 9ed6a84 and 5398e54.

📒 Files selected for processing (1)

• lib/crewai/src/crewai/llm.py

🚧 Files skipped from review as they are similar to previous changes (1)

• lib/crewai/src/crewai/llm.py

---

📝 Walkthrough

Walkthrough

lib/crewai/src/crewai/llm.py adds provider capability detection, extends LiteLLM introspection for reasoning support, exposes a get_capabilities() aggregator on LLM, and uses that result to validate response_format and reasoning_effort before API calls.

Changes

Provider Capability Detection

Layer / File(s)	Summary
ProviderCapabilities dataclass and provider allowlists lib/crewai/src/crewai/llm.py	Adds dataclass import, LiteLLM reasoning support plumbing, the ProviderCapabilities dataclass, and module-level provider allowlists for response_format and reasoning support.
LLM.get_capabilities() implementation lib/crewai/src/crewai/llm.py	Adds get_capabilities() to combine LiteLLM model info, provider allowlists, and existing helper methods into a ProviderCapabilities instance.
_validate_call_params reasoning and response_format guards lib/crewai/src/crewai/llm.py	Uses get_capabilities() to reject unsupported response_format and adds a reasoning_effort validation branch when reasoning support is absent.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: adding unified provider capability detection via ProviderCapabilities.
Linked Issues check	✅ Passed	The PR implements unified capabilities, preflight validation, and clearer errors for unsupported response_format and reasoning.
Out of Scope Changes check	✅ Passed	The changes stay focused on provider capability detection and related validation in llm.py.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/src/crewai/llm.py`:
- Line 2396: The early return that bails on missing LiteLLM introspection is
happening before the get_capabilities() method call on line 2396, which prevents
the fallback logic for response_format and reasoning_effort validation from
running. Move the early return statement to after the get_capabilities() call so
that the method's built-in fallback logic can execute before any conditional
exit occurs.
- Around line 2481-2485: The current logic unconditionally sets
supports_tool_calling to True when the provider is not in the
_PROVIDERS_WITHOUT_TOOL_CALLING blocklist, which discards valid False results
from supports_function_calling(). Replace the conditional logic that assigns
True on line 2485 with AND logic that checks both conditions: that the provider
is not in _PROVIDERS_WITHOUT_TOOL_CALLING AND that
self.supports_function_calling() returns True. This ensures the function's
negative results are respected instead of being overridden.
- Around line 2487-2488: The current code at line 2487-2488 uses only a
provider-level allowlist to determine reasoning support, which incorrectly
allows reasoning_effort for non-reasoning models under reasoning-capable
providers. Replace the static allowlist check for `supports_reasoning` with a
model-aware approach using `litellm.supports_reasoning(model,
custom_llm_provider=provider)` when available, and fall back to the static
`_PROVIDERS_WITH_REASONING` allowlist check only when that function is
unavailable or returns None. This ensures that reasoning_effort is only applied
to models that actually support reasoning, not just providers that support
reasoning for some of their models.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d127da02-d2a6-48dc-bd8a-73ba44d43884

📥 Commits

Reviewing files that changed from the base of the PR and between 854c67d and 77cfb72.

📒 Files selected for processing (1)

• lib/crewai/src/crewai/llm.py

[tcconnally]

Marking this ready for review. Summary of the CodeRabbit review pass (commit 5398e54):

Addressed (fixed):

1. Let get_capabilities() run before bailing on missing LiteLLM introspection (_validate_call_params) — the early return used or supports_response_schema is None, skipping validation even when litellm is installed. get_capabilities() already has a static-allowlist fallback for that case, so the guard was dropped to if not _ensure_litellm():.
2. Make reasoning detection model-aware — replaced the provider-only provider in _PROVIDERS_WITH_REASONING with litellm.supports_reasoning(model, custom_llm_provider=provider) plus static fallback, mirroring the existing response_format introspection pattern.

Rebutted (kept as-is, with maintainer-visible rationale, CodeRabbit agreed and withdrew):
3. Respect supports_function_calling() instead of overriding False with True — intentionally an optimistic-default + explicit _PROVIDERS_WITHOUT_TOOL_CALLING deny-list, because litellm.supports_function_calling() returns false negatives on stale/missing registry entries; switching to AND logic would silently disable tool calling for capable models. Mirrors the response_format fallback strategy.

Validated locally with ast.parse (full test run blocked by missing optional deps in this env, e.g. litellm/crewai_core). The two fixes follow the same introspection-first / static-fallback shape that the existing, tested response_format path uses.