Import AITURK IDE 1.0.0-beta.1 from Hermes 63279301; preserve MIT license
This commit is contained in:
@@ -0,0 +1,79 @@
|
||||
# providers/
|
||||
|
||||
Registry and ABC for every inference provider Hermes knows about.
|
||||
|
||||
Each provider is declared once as a `ProviderProfile`. Every other layer —
|
||||
auth resolution, transport kwargs, model listing, runtime routing — reads from
|
||||
these profiles instead of maintaining its own parallel data.
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
providers/
|
||||
├── base.py ProviderProfile dataclass + OMIT_TEMPERATURE sentinel
|
||||
├── __init__.py Registry: register_provider(), get_provider_profile(), list_providers()
|
||||
└── README.md This file
|
||||
```
|
||||
|
||||
The **profiles themselves** live as plugins under
|
||||
`plugins/model-providers/<name>/` (bundled in this repo) and
|
||||
`$HERMES_HOME/plugins/model-providers/<name>/` (per-user overrides). The
|
||||
registry in `providers/__init__.py` lazily discovers them the first time any
|
||||
consumer calls `get_provider_profile()` or `list_providers()`. See
|
||||
`plugins/model-providers/README.md` for the plugin contract and examples.
|
||||
|
||||
---
|
||||
|
||||
## How it wires in
|
||||
|
||||
The registry is populated on first access. After that, every downstream
|
||||
layer reads from it:
|
||||
|
||||
- `hermes_cli/auth.py` extends `PROVIDER_REGISTRY` with every api-key
|
||||
profile it sees (skipping `copilot`, `kimi-coding`, `kimi-coding-cn`,
|
||||
`zai`, `openrouter`, `custom` — those need bespoke token resolution).
|
||||
- `hermes_cli/models.py` extends `CANONICAL_PROVIDERS` and calls
|
||||
`profile.fetch_models()` inside `provider_model_ids()`.
|
||||
- `hermes_cli/doctor.py` adds a `/models` health check for each
|
||||
`auth_type="api_key"` profile.
|
||||
- `hermes_cli/config.py` injects every `env_var` into
|
||||
`OPTIONAL_ENV_VARS` so the setup wizard knows about it.
|
||||
- `hermes_cli/runtime_provider.py` reads `profile.api_mode` as a fallback
|
||||
when URL detection finds nothing.
|
||||
- `agent/model_metadata.py` maps hostname → provider via
|
||||
`profile.get_hostname()`.
|
||||
- `agent/auxiliary_client.py` reads `profile.default_aux_model` first
|
||||
before falling back to the legacy hardcoded dict.
|
||||
- `agent/transports/chat_completions.py::_build_kwargs_from_profile()`
|
||||
invokes `profile.prepare_messages()`, `profile.build_extra_body()`,
|
||||
and `profile.build_api_kwargs_extras()` on every call.
|
||||
- `run_agent.py` passes `provider_profile=<ProviderProfile>` so the
|
||||
transport takes the profile path instead of the legacy flag path.
|
||||
|
||||
---
|
||||
|
||||
## Adding a provider
|
||||
|
||||
See `plugins/model-providers/README.md` — drop a new directory there (or
|
||||
under `$HERMES_HOME/plugins/model-providers/` for a private plugin).
|
||||
|
||||
---
|
||||
|
||||
## Hooks you can override on `ProviderProfile`
|
||||
|
||||
| Hook | Purpose |
|
||||
|------|---------|
|
||||
| `get_hostname()` | URL-based detection — default derives from `base_url`. |
|
||||
| `prepare_messages(msgs)` | Provider-specific message preprocessing (Qwen normalises to list-of-parts, injects `cache_control`). |
|
||||
| `build_extra_body(**ctx)` | Provider-specific `extra_body` (OpenRouter provider prefs, Gemini `thinking_config`). |
|
||||
| `build_api_kwargs_extras(**ctx)` | `(extra_body_additions, top_level_kwargs)` — Kimi puts reasoning_effort top-level, Qwen splits `enable_thinking`/`thinking_budget`. |
|
||||
| `supported_reasoning_efforts(model)` | Declared per-model reasoning-effort vocabulary for gateways that 400 on unknown levels (Ramp Router reads its live catalog). `None` = defer to transport defaults, `()` = model takes no reasoning params, tuple = clamp target. Must be cache-only — called on the request hot path. |
|
||||
| `fetch_models(*, api_key)` | Live catalog fetch — default hits `{models_url or base_url}/models` with Bearer auth. Override for no-REST providers (Bedrock), OAuth catalogs (Anthropic), or public catalogs (OpenRouter). |
|
||||
|
||||
---
|
||||
|
||||
## Configuration fields
|
||||
|
||||
Full reference in `providers/base.py` dataclass definition.
|
||||
@@ -0,0 +1,414 @@
|
||||
"""Provider module registry.
|
||||
|
||||
Provider profiles can live in three places:
|
||||
|
||||
1. Bundled plugins: ``plugins/model-providers/<name>/`` (shipped with hermes-agent)
|
||||
2. User plugins: ``$HERMES_HOME/plugins/model-providers/<name>/``
|
||||
3. Pip-installed plugins: distributions exposing a ``hermes_agent.plugins``
|
||||
entry point (``module:func`` callable or a self-registering ``module``)
|
||||
|
||||
Each plugin directory contains:
|
||||
- ``__init__.py`` — calls ``register_provider(profile)`` at import
|
||||
- ``plugin.yaml`` — manifest (name, kind: model-provider, version, description)
|
||||
|
||||
Discovery is lazy: the first call to ``get_provider_profile()`` or
|
||||
``list_providers()`` scans both locations and imports every plugin. User
|
||||
plugins override bundled plugins on name collision (last-writer-wins), so
|
||||
third parties can monkey-patch or replace any built-in profile without
|
||||
editing the repo.
|
||||
|
||||
For backward compatibility, ``providers/*.py`` files (other than ``base.py``
|
||||
and ``__init__.py``) are still discovered via ``pkgutil.iter_modules``.
|
||||
This lets out-of-tree users drop a single-file profile into an editable
|
||||
install without the plugin dir structure. New profiles should prefer the
|
||||
plugin layout.
|
||||
|
||||
Usage::
|
||||
|
||||
from providers import get_provider_profile
|
||||
profile = get_provider_profile("nvidia") # ProviderProfile or None
|
||||
profile = get_provider_profile("kimi") # checks name + aliases
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import importlib.util
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from providers.base import OMIT_TEMPERATURE, ProviderProfile # noqa: F401
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_REGISTRY: dict[str, ProviderProfile] = {}
|
||||
_ALIASES: dict[str, str] = {}
|
||||
_PROVIDER_LIST_CACHE: list[ProviderProfile] | None = None
|
||||
_discovered = False
|
||||
|
||||
# Repo-root ``plugins/model-providers/`` — populated at discovery time.
|
||||
_BUNDLED_PLUGINS_DIR = (
|
||||
Path(__file__).resolve().parent.parent / "plugins" / "model-providers"
|
||||
)
|
||||
|
||||
|
||||
def register_provider(profile: ProviderProfile) -> None:
|
||||
"""Register a provider profile by name and aliases.
|
||||
|
||||
Later registrations with the same name replace earlier ones — so user
|
||||
plugins under ``$HERMES_HOME/plugins/model-providers/`` can override
|
||||
bundled profiles without editing repo code.
|
||||
"""
|
||||
global _PROVIDER_LIST_CACHE
|
||||
_REGISTRY[profile.name] = profile
|
||||
for alias in profile.aliases:
|
||||
_ALIASES[alias] = profile.name
|
||||
_PROVIDER_LIST_CACHE = None
|
||||
|
||||
|
||||
def get_provider_profile(name: str) -> ProviderProfile | None:
|
||||
"""Look up a provider profile by name or alias.
|
||||
|
||||
Returns None if the provider has no profile (falls back to generic).
|
||||
"""
|
||||
if not _discovered:
|
||||
_discover_providers()
|
||||
canonical = _ALIASES.get(name, name)
|
||||
return _REGISTRY.get(canonical)
|
||||
|
||||
|
||||
def list_providers() -> list[ProviderProfile]:
|
||||
"""Return all registered provider profiles (one per canonical name)."""
|
||||
global _PROVIDER_LIST_CACHE
|
||||
if not _discovered:
|
||||
_discover_providers()
|
||||
if _PROVIDER_LIST_CACHE is not None:
|
||||
return list(_PROVIDER_LIST_CACHE)
|
||||
# Deduplicate: _REGISTRY has canonical names; _ALIASES points to same objects
|
||||
seen: set[int] = set()
|
||||
result: list[ProviderProfile] = []
|
||||
for profile in _REGISTRY.values():
|
||||
pid = id(profile)
|
||||
if pid not in seen:
|
||||
seen.add(pid)
|
||||
result.append(profile)
|
||||
_PROVIDER_LIST_CACHE = result
|
||||
return list(result)
|
||||
|
||||
|
||||
def _user_plugins_dir() -> Path | None:
|
||||
"""Return ``$HERMES_HOME/plugins/model-providers/`` if it exists."""
|
||||
try:
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
d = get_hermes_home() / "plugins" / "model-providers"
|
||||
return d if d.is_dir() else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _installed_plugins_dir() -> Path | None:
|
||||
"""Return ``$HERMES_HOME/plugins/`` if it exists.
|
||||
|
||||
This is where ``hermes plugins install`` clones a plugin — flat, one
|
||||
directory per plugin, NOT under ``model-providers/``. See
|
||||
:func:`_discover_installed_provider_plugins`.
|
||||
"""
|
||||
try:
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
d = get_hermes_home() / "plugins"
|
||||
return d if d.is_dir() else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _declares_model_provider_kind(plugin_dir: Path) -> bool:
|
||||
"""Whether ``plugin_dir``'s manifest declares ``kind: model-provider``.
|
||||
|
||||
Only that kind is imported from the flat install directory — every other
|
||||
plugin there belongs to ``PluginManager``, which owns its lifecycle and
|
||||
consent flow. Parsed with PyYAML when available, falling back to a line
|
||||
scan so provider discovery never hard-depends on it.
|
||||
"""
|
||||
for filename in ("plugin.yaml", "plugin.yml"):
|
||||
manifest = plugin_dir / filename
|
||||
if not manifest.is_file():
|
||||
continue
|
||||
try:
|
||||
text = manifest.read_text(encoding="utf-8", errors="replace")
|
||||
except Exception:
|
||||
return False
|
||||
try:
|
||||
import yaml
|
||||
|
||||
data = yaml.safe_load(text)
|
||||
if isinstance(data, dict):
|
||||
return str(data.get("kind", "")).strip() == "model-provider"
|
||||
except Exception:
|
||||
pass
|
||||
for line in text.splitlines():
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("#") or ":" not in stripped:
|
||||
continue
|
||||
key, _, value = stripped.partition(":")
|
||||
if key.strip() == "kind":
|
||||
return value.strip().strip("\"'") == "model-provider"
|
||||
return False
|
||||
return False
|
||||
|
||||
|
||||
def _import_plugin_dir(plugin_dir: Path, source: str) -> None:
|
||||
"""Import a single plugin directory so it self-registers.
|
||||
|
||||
``source`` is "bundled" or "user", used only for log messages.
|
||||
"""
|
||||
init_file = plugin_dir / "__init__.py"
|
||||
if not init_file.exists():
|
||||
return
|
||||
|
||||
# Give bundled plugins a stable import path (``plugins.model_providers.<name>``)
|
||||
# so relative imports within the plugin work. User plugins load via
|
||||
# ``importlib.util.spec_from_file_location`` with a unique module name so
|
||||
# multiple HERMES_HOME profiles don't alias each other.
|
||||
safe_name = plugin_dir.name.replace("-", "_")
|
||||
if source == "bundled":
|
||||
module_name = f"plugins.model_providers.{safe_name}"
|
||||
else:
|
||||
module_name = f"_hermes_user_provider_{safe_name}"
|
||||
|
||||
if module_name in sys.modules:
|
||||
return # already imported
|
||||
|
||||
try:
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
module_name, init_file, submodule_search_locations=[str(plugin_dir)]
|
||||
)
|
||||
if spec is None or spec.loader is None:
|
||||
return
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[module_name] = module
|
||||
spec.loader.exec_module(module)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"Failed to load %s provider plugin %s: %s", source, plugin_dir.name, exc
|
||||
)
|
||||
sys.modules.pop(module_name, None)
|
||||
|
||||
|
||||
def _discover_entry_point_providers() -> None:
|
||||
"""Import pip-installed provider plugins via the ``hermes_agent.plugins``
|
||||
entry-point group so they self-register.
|
||||
|
||||
A distribution ships::
|
||||
|
||||
[project.entry-points."hermes_agent.plugins"]
|
||||
acme-inference = "acme_hermes_plugin:register"
|
||||
|
||||
The target may be either a **callable** (``module:func`` — invoked with no
|
||||
args; typically calls ``register_provider(profile)``) or a **module**
|
||||
(``module`` — imported for its module-level ``register_provider`` side
|
||||
effect, mirroring the directory-plugin ``__init__.py`` contract).
|
||||
|
||||
Gating and safety:
|
||||
|
||||
* **Opt-in.** Entry-point plugins are subject to the same
|
||||
``plugins.enabled`` allow-list (and ``plugins.disabled`` deny-list) the
|
||||
general PluginManager enforces — a pip package is never imported just
|
||||
because it is installed. An entry point whose name is not enabled is
|
||||
skipped without loading.
|
||||
* **Provider targets only.** The ``hermes_agent.plugins`` group is shared
|
||||
with general plugins whose target is ``register(ctx)``. Callables that
|
||||
require arguments are skipped here (the PluginManager owns them);
|
||||
provider registration hooks take no arguments by contract.
|
||||
|
||||
Failures are swallowed per-entry (a broken third-party package must not
|
||||
break provider discovery) and logged at warning level. This scan runs
|
||||
first, so filesystem plugins (bundled + ``$HERMES_HOME``) keep their
|
||||
documented override precedence via last-writer-wins in
|
||||
``register_provider()`` — a pip package cannot hijack a first-party
|
||||
provider name.
|
||||
"""
|
||||
try:
|
||||
import importlib.metadata as _md
|
||||
except Exception: # pragma: no cover — importlib.metadata always present ≥3.8
|
||||
return
|
||||
|
||||
# Same opt-in gate as the general PluginManager: only entry points named
|
||||
# in ``plugins.enabled`` load, and ``plugins.disabled`` always wins.
|
||||
try:
|
||||
from hermes_cli.plugins import _get_disabled_plugins, _get_enabled_plugins
|
||||
|
||||
enabled = _get_enabled_plugins() # None = nothing enabled yet (opt-in default)
|
||||
disabled = _get_disabled_plugins()
|
||||
except Exception: # pragma: no cover — config layer unavailable
|
||||
enabled, disabled = None, set()
|
||||
if not enabled:
|
||||
return
|
||||
|
||||
group = "hermes_agent.plugins"
|
||||
try:
|
||||
eps = _md.entry_points()
|
||||
# Python 3.10+ exposes .select(); older returns a dict-like mapping.
|
||||
if hasattr(eps, "select"):
|
||||
group_eps = list(eps.select(group=group))
|
||||
else: # pragma: no cover — legacy interpreters
|
||||
group_eps = list(eps.get(group, [])) # type: ignore[attr-defined]
|
||||
except Exception as exc:
|
||||
logger.debug("entry-point provider scan skipped: %s", exc)
|
||||
return
|
||||
|
||||
for ep in group_eps:
|
||||
if ep.name not in enabled or ep.name in disabled:
|
||||
logger.debug(
|
||||
"entry-point provider %r skipped: not enabled in config", ep.name
|
||||
)
|
||||
continue
|
||||
try:
|
||||
loaded = ep.load()
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"Failed to load entry-point provider plugin %r: %s", ep.name, exc
|
||||
)
|
||||
continue
|
||||
# ``module:func`` → callable we invoke; bare ``module`` → import side
|
||||
# effect already happened during load(). Only call when it's callable
|
||||
# AND zero-arg: general plugins in this shared group expose
|
||||
# ``register(ctx)`` (requires an argument) and belong to the
|
||||
# PluginManager, not the provider registry.
|
||||
if callable(loaded):
|
||||
if _requires_arguments(loaded):
|
||||
logger.debug(
|
||||
"entry-point %r skipped by provider scan: target requires "
|
||||
"arguments (general plugin owned by PluginManager)",
|
||||
ep.name,
|
||||
)
|
||||
continue
|
||||
try:
|
||||
loaded()
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"Entry-point provider plugin %r raised on invocation: %s",
|
||||
ep.name,
|
||||
exc,
|
||||
)
|
||||
|
||||
|
||||
def _requires_arguments(fn) -> bool:
|
||||
"""True when ``fn`` cannot be called with zero arguments.
|
||||
|
||||
Used to distinguish provider registration hooks (zero-arg by contract)
|
||||
from general plugin hooks (``register(ctx)``) sharing the same entry-point
|
||||
group. Unintrospectable callables (C extensions) are treated as zero-arg
|
||||
and left to the per-entry exception guard.
|
||||
"""
|
||||
import inspect
|
||||
|
||||
try:
|
||||
sig = inspect.signature(fn)
|
||||
except (TypeError, ValueError): # pragma: no cover — builtins/C callables
|
||||
return False
|
||||
for param in sig.parameters.values():
|
||||
if param.kind in (
|
||||
inspect.Parameter.POSITIONAL_ONLY,
|
||||
inspect.Parameter.POSITIONAL_OR_KEYWORD,
|
||||
inspect.Parameter.KEYWORD_ONLY,
|
||||
) and param.default is inspect.Parameter.empty:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _discover_providers() -> None:
|
||||
"""Populate the registry by importing every provider plugin.
|
||||
|
||||
Order:
|
||||
1. Bundled plugins at ``<repo>/plugins/model-providers/<name>/``
|
||||
2. User plugins at ``$HERMES_HOME/plugins/model-providers/<name>/``
|
||||
2b. Plugins installed by ``hermes plugins install`` at
|
||||
``$HERMES_HOME/plugins/<name>/`` that declare ``kind: model-provider``
|
||||
3. Legacy per-file modules at ``providers/<name>.py`` (back-compat)
|
||||
|
||||
Each step imports its plugins, which call ``register_provider()`` at
|
||||
module-level. Later steps win on name collision.
|
||||
"""
|
||||
global _discovered
|
||||
if _discovered:
|
||||
return
|
||||
_discovered = True
|
||||
|
||||
# 0. Pip-installed plugins — entry points in the ``hermes_agent.plugins``
|
||||
# group (the same group the general PluginManager uses). The manager
|
||||
# records model-provider manifests for introspection but deliberately
|
||||
# does NOT import them — provider lifecycle is owned here — so without
|
||||
# this step a ``pip install``ed provider never calls
|
||||
# ``register_provider()`` and is never selectable.
|
||||
#
|
||||
# Discovered FIRST, i.e. lowest precedence: because
|
||||
# ``register_provider()`` is last-writer-wins, running this before the
|
||||
# filesystem steps means a bundled or ``$HERMES_HOME`` profile of the
|
||||
# same name always overrides a pip-installed one. That prevents a
|
||||
# third-party package from silently hijacking a first-party provider
|
||||
# name (e.g. ``openrouter``) while still letting pip packages add
|
||||
# genuinely new providers.
|
||||
_discover_entry_point_providers()
|
||||
|
||||
# 1. Bundled plugins — shipped with hermes-agent.
|
||||
if _BUNDLED_PLUGINS_DIR.is_dir():
|
||||
for child in sorted(_BUNDLED_PLUGINS_DIR.iterdir()):
|
||||
if not child.is_dir() or child.name.startswith(("_", ".")):
|
||||
continue
|
||||
_import_plugin_dir(child, "bundled")
|
||||
|
||||
# 2. User plugins — under $HERMES_HOME/plugins/model-providers/<name>/.
|
||||
# These can override any bundled profile of the same name (last-writer-wins
|
||||
# in register_provider()).
|
||||
user_dir = _user_plugins_dir()
|
||||
if user_dir is not None:
|
||||
for child in sorted(user_dir.iterdir()):
|
||||
if not child.is_dir() or child.name.startswith(("_", ".")):
|
||||
continue
|
||||
_import_plugin_dir(child, "user")
|
||||
|
||||
# 2b. Plugins installed by ``hermes plugins install`` / the plugin index.
|
||||
# Those clone into $HERMES_HOME/plugins/<name>/ — flat, NOT under
|
||||
# model-providers/ — so step 2 never sees them. PluginManager does not
|
||||
# import them either: it classifies ``kind: model-provider`` and routes
|
||||
# it here on purpose. Without this step the documented install path
|
||||
# silently half-works — the CLI reports success and the provider does
|
||||
# not exist. Only manifests declaring that kind are imported; every
|
||||
# other plugin in this directory belongs to PluginManager.
|
||||
installed_dir = _installed_plugins_dir()
|
||||
if installed_dir is not None:
|
||||
for child in sorted(installed_dir.iterdir()):
|
||||
if not child.is_dir() or child.name.startswith(("_", ".")):
|
||||
continue
|
||||
if child.name == "model-providers":
|
||||
continue # handled by step 2
|
||||
if not _declares_model_provider_kind(child):
|
||||
continue
|
||||
_import_plugin_dir(child, "user")
|
||||
|
||||
# 3. Legacy single-file profiles at providers/<name>.py. Kept for
|
||||
# back-compat — if someone drops a ``providers/foo.py`` into an
|
||||
# editable install, it still works without the plugin layout.
|
||||
try:
|
||||
import pkgutil
|
||||
|
||||
import providers as _pkg
|
||||
|
||||
for _importer, modname, _ispkg in pkgutil.iter_modules(_pkg.__path__):
|
||||
if modname.startswith("_") or modname == "base":
|
||||
continue
|
||||
try:
|
||||
importlib.import_module(f"providers.{modname}")
|
||||
except ImportError as exc:
|
||||
logger.warning(
|
||||
"Failed to import legacy provider module %s: %s", modname, exc
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# (Pip entry-point providers are discovered in step 0, before the
|
||||
# filesystem plugins, so first-party profiles always win on name
|
||||
# collision — see _discover_entry_point_providers.)
|
||||
@@ -0,0 +1,332 @@
|
||||
"""Provider profile base class.
|
||||
|
||||
A ProviderProfile declares everything about an inference provider in one place:
|
||||
auth, endpoints, client quirks, request-time quirks. The transport reads this
|
||||
instead of receiving 20+ boolean flags.
|
||||
|
||||
Provider profiles are DECLARATIVE — they describe the provider's behavior.
|
||||
They do NOT own client construction, credential rotation, or streaming.
|
||||
Those stay on AIAgent.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Sentinel for "omit temperature entirely" (Kimi: server manages it)
|
||||
OMIT_TEMPERATURE = object()
|
||||
|
||||
|
||||
def _profile_user_agent() -> str:
|
||||
"""Return a ``hermes-cli/<version>`` UA string, with a stable fallback.
|
||||
|
||||
Used by ``ProviderProfile.fetch_models`` so the catalog probe is not
|
||||
served the default ``Python-urllib/<ver>`` UA — some providers
|
||||
(OpenCode Zen, etc.) sit behind a WAF that returns 403 for that.
|
||||
"""
|
||||
try:
|
||||
from hermes_cli import __version__ as _ver # lazy: avoid layer cycle at import time
|
||||
return f"hermes-cli/{_ver}"
|
||||
except Exception:
|
||||
return "hermes-cli"
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProviderProfile:
|
||||
"""Base provider profile — subclass or instantiate with overrides."""
|
||||
|
||||
# ── Identity ─────────────────────────────────────────────
|
||||
name: str
|
||||
api_mode: str = "chat_completions"
|
||||
aliases: tuple = ()
|
||||
|
||||
# ── Human-readable metadata ───────────────────────────────
|
||||
display_name: str = "" # e.g. "GMI Cloud" — shown in picker/labels
|
||||
description: str = "" # e.g. "GMI Cloud (multi-model direct API)" — picker subtitle
|
||||
signup_url: str = "" # e.g. "https://www.gmicloud.ai/" — shown during setup
|
||||
|
||||
# ── Auth & endpoints ─────────────────────────────────────
|
||||
env_vars: tuple = ()
|
||||
base_url: str = ""
|
||||
models_url: str = "" # explicit models endpoint; falls back to {base_url}/models
|
||||
auth_type: str = "api_key" # api_key|oauth_device_code|oauth_external|copilot|aws_sdk
|
||||
supports_health_check: bool = True # False → doctor skips /models probe for this provider
|
||||
|
||||
# ── Vision support ────────────────────────────────────────
|
||||
# True when the provider's API accepts image content inside
|
||||
# tool-result messages natively. Set on providers that expose
|
||||
# multimodal models via tool results (Anthropic Messages API,
|
||||
# OpenAI Chat Completions, Gemini, MiniMax, etc.).
|
||||
# Falls back to model-catalog lookup when False and the provider
|
||||
# has no registered profile.
|
||||
supports_vision: bool = False
|
||||
|
||||
# True when the provider's API accepts list-type tool message
|
||||
# content (multipart with image_url parts). Defaults to True for
|
||||
# backward compatibility. Set to False for providers that accept
|
||||
# multimodal user messages but reject list-type tool content
|
||||
# (e.g. Xiaomi MiMo, which returns 400 "text is not set").
|
||||
supports_vision_tool_messages: bool = True
|
||||
|
||||
# True only when this provider's Chat Completions endpoint explicitly
|
||||
# documents ``prompt_cache_key`` as an accepted request body field. This
|
||||
# is deliberately opt-in: many OpenAI-compatible endpoints reject unknown
|
||||
# top-level fields rather than ignoring them.
|
||||
supports_prompt_cache_key: bool = False
|
||||
|
||||
# ── External-process providers (auth_type="external_process") ──
|
||||
# An agent CLI driven over stdio (ACP) rather than an HTTP endpoint. These
|
||||
# describe how to launch it; hermes_cli/auth.py's
|
||||
# resolve_external_process_provider_credentials() reads them instead of
|
||||
# hardcoding one vendor's binary. Env vars are checked in order and win
|
||||
# over the static defaults, so an operator can point at a custom build.
|
||||
process_command: str = "" # default binary, e.g. "copilot"
|
||||
process_args: tuple = () # default argv tail, e.g. ("--acp", "--stdio")
|
||||
process_command_env_vars: tuple = () # env overrides for the binary, in priority order
|
||||
process_args_env_var: str = "" # env override for argv (shlex-split)
|
||||
|
||||
# ── Model catalog ─────────────────────────────────────────
|
||||
# fallback_models: curated list shown in /model picker when live fetch fails.
|
||||
# Only agentic models that support tool calling should appear here.
|
||||
fallback_models: tuple = ()
|
||||
|
||||
# hostname: base hostname for URL→provider reverse-mapping in model_metadata.py
|
||||
# e.g. "api.gmi-serving.com". Derived from base_url when empty.
|
||||
hostname: str = ""
|
||||
|
||||
# ── Client-level quirks (set once at client construction) ─
|
||||
default_headers: dict[str, str] = field(default_factory=dict)
|
||||
|
||||
# ── Request-level quirks ─────────────────────────────────
|
||||
# Temperature: None = use caller's default, OMIT_TEMPERATURE = don't send
|
||||
fixed_temperature: Any = None
|
||||
default_max_tokens: int | None = None
|
||||
default_aux_model: str = (
|
||||
"" # cheap model for auxiliary tasks (compression, vision, etc.)
|
||||
)
|
||||
# empty = use main model
|
||||
|
||||
# ── Hooks (override in subclass for complex providers) ───
|
||||
|
||||
def resolve_aux_model(self, *, vision: bool = False) -> str:
|
||||
"""Return a LIVE cheap-model id for auxiliary tasks, or "".
|
||||
|
||||
``default_aux_model`` is a hardcoded id in source, so it rots: when the
|
||||
provider retires that model every auxiliary call spends a round-trip
|
||||
404ing before the retry net catches it. Providers that publish a
|
||||
machine-readable recommendation should override this and query it, so
|
||||
the cheap tier tracks the upstream catalog instead of a constant a human
|
||||
has to remember to bump.
|
||||
|
||||
Contract: cheap to call (implementations must cache — this runs on
|
||||
client-resolution paths), never raises, and returns "" when it has no
|
||||
answer so the caller falls through to ``default_aux_model``.
|
||||
"""
|
||||
return ""
|
||||
|
||||
def get_hostname(self) -> str:
|
||||
"""Return the provider's base hostname for URL-based detection.
|
||||
|
||||
Uses self.hostname if set explicitly, otherwise derives it from base_url.
|
||||
e.g. 'https://api.gmi-serving.com/v1' → 'api.gmi-serving.com'
|
||||
"""
|
||||
if self.hostname:
|
||||
return self.hostname
|
||||
if self.base_url:
|
||||
from urllib.parse import urlparse
|
||||
return urlparse(self.base_url).hostname or ""
|
||||
return ""
|
||||
|
||||
def prepare_messages(self, messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
"""Provider-specific message preprocessing.
|
||||
|
||||
Called AFTER codex field sanitization, BEFORE developer role swap.
|
||||
Default: pass-through.
|
||||
"""
|
||||
return messages
|
||||
|
||||
def build_extra_body(
|
||||
self, *, session_id: str | None = None, **context: Any
|
||||
) -> dict[str, Any]:
|
||||
"""Provider-specific extra_body fields.
|
||||
|
||||
Merged into the API kwargs extra_body. Default: empty dict.
|
||||
"""
|
||||
return {}
|
||||
|
||||
def build_api_kwargs_extras(
|
||||
self,
|
||||
*,
|
||||
reasoning_config: dict | None = None,
|
||||
**context: Any,
|
||||
) -> tuple[dict[str, Any], dict[str, Any]]:
|
||||
"""Provider-specific kwargs split between extra_body and top-level api_kwargs.
|
||||
|
||||
Returns (extra_body_additions, top_level_kwargs).
|
||||
The transport merges extra_body_additions into extra_body, and
|
||||
top_level_kwargs directly into api_kwargs.
|
||||
|
||||
This split exists because some providers put reasoning config in
|
||||
extra_body (OpenRouter: extra_body.reasoning) while others put it
|
||||
as top-level api_kwargs (Kimi: api_kwargs.reasoning_effort).
|
||||
|
||||
Default: ({}, {}).
|
||||
"""
|
||||
return {}, {}
|
||||
|
||||
def default_vision_model(self) -> str | None:
|
||||
"""Return a default vision model id for this provider, or None.
|
||||
|
||||
Overrideable hook for providers that discover their vision default at
|
||||
runtime (e.g. from a live catalog) rather than pinning one in code.
|
||||
Keeps provider-specific vision discovery inside the provider's plugin
|
||||
instead of a name-check branch in shared vision resolution.
|
||||
|
||||
Default: None (no provider-specific vision model — the caller falls
|
||||
back to the user's chat model or the aggregator chain).
|
||||
"""
|
||||
return None
|
||||
|
||||
def get_max_tokens(self, model: str | None) -> int | None:
|
||||
"""Return the default max_tokens cap for *model*.
|
||||
|
||||
Overrideable hook for providers that need per-model output caps —
|
||||
e.g. a relay that fronts several upstream backends, each with a
|
||||
different completion-token limit. The transport calls this when
|
||||
the user hasn't set an explicit max_tokens.
|
||||
|
||||
Default: return self.default_max_tokens (the static profile field),
|
||||
ignoring the model name. Override in a subclass to vary the cap
|
||||
per-model.
|
||||
"""
|
||||
return self.default_max_tokens
|
||||
|
||||
def supported_reasoning_efforts(
|
||||
self, model: str | None
|
||||
) -> tuple[str, ...] | None:
|
||||
"""Declared reasoning-effort vocabulary for *model* on this provider.
|
||||
|
||||
Overrideable hook for providers whose gateway validates
|
||||
``reasoning.effort`` per model instead of ignoring or clamping
|
||||
unknown levels server-side (Ramp Router derives this from its live
|
||||
``/v1/models`` catalog). The Responses transport consults it before
|
||||
falling back to its built-in per-backend vocabularies; it is the
|
||||
profile-declared analog of the OpenRouter catalog clamp on the
|
||||
chat-completions path (``openrouter_model_reasoning_capabilities``).
|
||||
|
||||
Tri-state contract:
|
||||
- ``None`` — unknown/undeclared: the transport keeps its default
|
||||
vocabulary for the wire (this base implementation).
|
||||
- ``()`` — the model accepts NO reasoning parameters at all; the
|
||||
transport must omit reasoning fields entirely (some gateways
|
||||
return HTTP 400 rather than ignoring them).
|
||||
- non-empty tuple — clamp the requested effort onto these levels
|
||||
(``agent.reasoning_effort.clamp_effort`` semantics: nearest
|
||||
weaker supported level, never escalate).
|
||||
|
||||
Implementations are called on the per-request hot path and must not
|
||||
block on network I/O — answer from a cache and return None while
|
||||
cold.
|
||||
"""
|
||||
return None
|
||||
|
||||
def create_client(self, **client_kwargs: Any) -> Any | None:
|
||||
"""Return a provider-specific client, or ``None`` for the standard one.
|
||||
|
||||
Most providers speak OpenAI-compatible HTTP and want the shared
|
||||
``openai.OpenAI`` client the core builds — they inherit this and return
|
||||
``None``. A provider whose wire protocol is not HTTP at all (the ACP
|
||||
subprocess shims) or which needs a native SDK overrides this and
|
||||
returns its own client object.
|
||||
|
||||
``client_kwargs`` is the same mapping the core would have passed to
|
||||
``openai.OpenAI`` (``api_key``, ``base_url``, ``command``, ``args``,
|
||||
timeouts, headers…). Unknown keys must be tolerated: the core adds to
|
||||
this mapping over time, so an override should accept ``**kwargs`` and
|
||||
pick what it needs rather than enumerate.
|
||||
|
||||
Returning ``None`` (the default) is always safe — the caller falls
|
||||
through to its existing construction path.
|
||||
|
||||
This is the hook that lets a provider ship *outside* this tree: with it,
|
||||
a profile registered from ``~/.hermes/plugins/model-providers/`` or a
|
||||
pip entry point can supply its own transport without any core edit. See
|
||||
``plugins/model-providers/copilot-acp/`` for the in-tree example.
|
||||
"""
|
||||
return None
|
||||
|
||||
def fetch_models(
|
||||
self,
|
||||
*,
|
||||
api_key: str | None = None,
|
||||
base_url: str | None = None,
|
||||
timeout: float = 8.0,
|
||||
) -> list[str] | None:
|
||||
"""Fetch the live model list from the provider's models endpoint.
|
||||
|
||||
Returns a list of model ID strings, or None if the fetch failed or
|
||||
the provider does not support live model listing.
|
||||
|
||||
Resolution order for the endpoint URL:
|
||||
1. base_url + "/models", but ONLY when the caller passed a base_url
|
||||
that differs from this profile's default (a user-configured
|
||||
model.base_url pointing at a proxy/custom endpoint). Callers
|
||||
pass base_url unconditionally — falling back to the profile
|
||||
default when the user configured nothing — so equality with
|
||||
self.base_url means "not customised" and must not shadow
|
||||
models_url.
|
||||
2. self.models_url (explicit override — use when the models
|
||||
endpoint differs from the inference base URL, e.g. OpenRouter
|
||||
exposes a public catalog at /api/v1/models while inference is
|
||||
at /api/v1)
|
||||
3. self.base_url + "/models" (standard OpenAI-compat fallback)
|
||||
|
||||
The default implementation sends Bearer auth when api_key is given
|
||||
and forwards self.default_headers. Override to customise auth, path,
|
||||
response shape, or to return None for providers with no REST catalog.
|
||||
|
||||
Callers must always fall back to the static _PROVIDER_MODELS list
|
||||
when this returns None.
|
||||
"""
|
||||
caller_base = (base_url or "").strip()
|
||||
effective_base = caller_base or self.base_url
|
||||
custom_base = bool(caller_base) and (
|
||||
caller_base.rstrip("/") != (self.base_url or "").rstrip("/")
|
||||
)
|
||||
if custom_base:
|
||||
url = caller_base.rstrip("/") + "/models"
|
||||
else:
|
||||
url = (self.models_url or "").strip()
|
||||
if not url:
|
||||
if not effective_base:
|
||||
return None
|
||||
url = effective_base.rstrip("/") + "/models"
|
||||
|
||||
import json
|
||||
import urllib.request
|
||||
|
||||
from hermes_cli.urllib_security import open_credentialed_url
|
||||
|
||||
req = urllib.request.Request(url)
|
||||
if api_key:
|
||||
req.add_header("Authorization", f"Bearer {api_key}")
|
||||
req.add_header("Accept", "application/json")
|
||||
# Some providers (e.g. OpenCode Zen) sit behind a WAF that blocks
|
||||
# the default ``Python-urllib/<ver>`` User-Agent. Set a generic
|
||||
# hermes-cli UA so the catalog endpoint is reachable.
|
||||
req.add_header("User-Agent", _profile_user_agent())
|
||||
for k, v in self.default_headers.items():
|
||||
req.add_header(k, v)
|
||||
|
||||
try:
|
||||
with open_credentialed_url(req, timeout=timeout) as resp:
|
||||
data = json.loads(resp.read().decode())
|
||||
items = data if isinstance(data, list) else data.get("data", [])
|
||||
return [m["id"] for m in items if isinstance(m, dict) and "id" in m]
|
||||
except Exception as exc:
|
||||
logger.debug("fetch_models(%s): %s", self.name, exc)
|
||||
return None
|
||||
Reference in New Issue
Block a user