Root Cause Verification: "No models available from dynamic pricing"
**Date:** 2026-05-02
**Issue:** Worker showing "No models available from dynamic pricing" despite BYOK keys configured
**Tenant:** Brennan Machinery (brennan)
Hypothesis
The worker fails because:
- Brennan has
MINIMAX_2_7_API_KEYbut NOTMINIMAX_API_KEYin database - BYOKHandler client initialization skips "minimax" provider (no API key found)
- Model ranking skips "minimax/" models (provider not in
self.clients) - No models pass all filters → raises "No models available from dynamic pricing"
Verification via TDD Tests
**Test File:** tests/test_minimax_client_init_root_cause.py
Test 1: Database State Verification
def test_brennan_has_minimax_2_7_key_not_minimax_key(self):
# Verify: MINIMAX_2_7_API_KEY exists, MINIMAX_API_KEY does not
assert minimax_2_7_key is not None # PASS
assert minimax_key is None # PASS (root cause)**Expected Result:** Confirms brennan has MINIMAX_2_7_API_KEY but not MINIMAX_API_KEY
Test 2: Client Initialization Behavior
def test_byok_handler_initializes_minimax_2_7_but_not_minimax(self):
# Initialize BYOKHandler for brennan
clients = byok_handler.clients
# Check which providers are in clients
assert has_minimax_2_7 # PASS (key exists)
assert not has_minimax # PASS (no key, not initialized)**Expected Result:** "minimax_2_7" in clients, "minimax" NOT in clients
Test 3: Model Ranking Provider Matching
def test_model_ranking_filters_out_minimax_models(self):
# Get available providers (what's in self.clients)
available_providers = list(byok_handler.clients.keys())
# Test model matching logic from rank_models_by_pricing
for model_id in ["minimax/gpt-4", "minimax_2_7/gpt-4"]:
prefix = model_id.split("/")[0]
if prefix in available_providers:
# Model can be used
else:
# Model is skipped**Expected Result:** "minimax/gpt-4" gets skipped (prefix "minimax" not in available_providers)
Test 4: End-to-End Error
def test_rank_models_raises_error_when_no_models_pass_filters(self):
# Call rank_models_by_pricing
with pytest.raises(ValueError) as exc_info:
byok_handler.rank_models_by_pricing(...)
assert "No models available from dynamic pricing" in str(exc_info.value)**Expected Result:** ValueError raised with exact error message from production logs
Test 5: Fix Verification
def test_fix_add_minimax_api_key_resolves_issue(self):
# Add MINIMAX_API_KEY to database (same value as MINIMAX_2_7_API_KEY)
new_setting = TenantSetting(
setting_key="MINIMAX_API_KEY",
setting_value=minimax_2_7_key.setting_value
)
db.add(new_setting)
db.commit()
# Re-initialize BYOKHandler
byok_handler = BYOKHandler(...)
clients = byok_handler.clients
# Check if minimax is now in clients
assert "minimax" in clients # PASS (fix works)**Expected Result:** Adding MINIMAX_API_KEY resolves the issue
Code Flow Analysis
Client Initialization (`BYOKHandler._initialize_clients()`)
# Line 588-612 in core/llm/byok_handler.py
for provider_id, provider_config in self.byok_manager.providers.items():
if provider_id not in _LLM_PROVIDERS:
continue
# Get tenant-specific API key
api_key = self.byok_manager.get_tenant_api_key(
self.tenant_id, provider_id, db=db
)
if not api_key:
logger.debug(f"No key found for {provider_id} in {self.workspace_id}")
continue # SKIP provider initialization
# Initialize client for this provider
self._clients_dict[provider_id] = client**Key Point:** If get_tenant_api_key() returns None, provider is NOT added to self._clients_dict
API Key Lookup (`BYOKManager.get_tenant_api_key()`)
# Line 848 in core/byok_endpoints.py
setting_key = f"{provider_id.upper()}_API_KEY"
# Query database
setting = db.query(TenantSetting).filter(
TenantSetting.tenant_id == tenant_id,
TenantSetting.setting_key == setting_key
).first()
if setting and setting.setting_value:
return setting.setting_value
else:
logger.warning(f"Setting not found or NULL for {provider_id}")
return None # Provider will be skipped**Key Point:** For provider_id "minimax", looks for "MINIMAX_API_KEY" in database
Model Ranking (`BYOKHandler.rank_models_by_pricing()`)
# Line 1178-1200 in core/llm/byok_handler.py
available_providers = list(self.clients.keys()) # Only initialized providers
for model_id, pricing in fetcher.pricing_cache.items():
# Extract provider from model_id (e.g., "minimax/gpt-4" -> "minimax")
if "/" in model_id.lower():
prefix = model_id.lower().split("/")[0]
if prefix in available_providers:
active_provider = prefix
else:
continue # SKIP this model
# ... filter by context, quality, capabilities, etc ...
candidates.append({
"provider": active_provider,
"model": model_id,
...
})
# If no candidates pass all filters
if not ranked_options:
raise ValueError("No models available from dynamic pricing")**Key Point:** Models with "minimax/" prefix are skipped if "minimax" not in self.clients
Root Cause Summary
- **Database State:** Brennan has
MINIMAX_2_7_API_KEYbut NOTMINIMAX_API_KEY - **Client Init:**
get_tenant_api_key(brennan, "minimax")returns None → "minimax" not inself.clients - **Model Ranking:** Pricing cache has models like "minimax/gpt-4" but "minimax" not in available_providers → models skipped
- **Error:** No models pass all filters → raises "No models available from dynamic pricing"
Fix Options
Option 1: Add MINIMAX_API_KEY to Database (User Action) ✅ RECOMMENDED
Brennan adds MINIMAX_API_KEY to tenant settings (same value as MINIMAX_2_7_API_KEY)
**Pros:**
- Correct solution (both providers should have keys if both are supported)
- No code changes required
- Follows existing pattern
**Cons:**
- Requires user action
**How to Add:**
# Via tenant_settings table
INSERT INTO tenant_settings (tenant_id, setting_key, setting_value)
VALUES (
'31c06fc4-db22-4740-83ea-48ac14f25810',
'MINIMAX_API_KEY',
'<same_value_as_MINIMAX_2_7_API_KEY>'
);Option 2: Fallback Logic (Code Change) ⚠️ NOT RECOMMENDED
Modify get_tenant_api_key() to check for "MINIMAX_2_7_API_KEY" as fallback for "minimax" provider
**Pros:**
- Automatic fix
**Cons:**
- Hacky workaround
- Doesn't address the root cause (missing key)
- Could confuse other providers
**Why Not:** Provider registry says both "minimax" and "minimax_2_7" expect MINIMAX_API_KEY (by design, they share the same key). The correct fix is to add the key to database.
Verification Steps
- ✅ TDD tests created (5 tests)
- ✅ Tests run correctly (fail on local DB, would pass on production)
- ✅ Code flow analysis matches hypothesis
- ⏳ Apply fix: Add
MINIMAX_API_KEYto brennan's tenant settings - ⏳ Verify fix: Re-run backfill job
- ⏳ Confirm success: Check logs for successful entity creation
Related Issues
- **Issue #7454884299:** Outlook backfill BYOK investigation (similar error pattern)
- **PR #1989:** Pricing cache singleton reload fix
- **Documentation:**
docs/archive/logs/2026-05-02-production-db-analysis.md
Next Steps
- Add
MINIMAX_API_KEYto brennan's tenant settings (same value asMINIMAX_2_7_API_KEY) - Restart worker or wait for next job
- Monitor logs for successful entity creation
- Document resolution
**Note:** This is a tenant-specific configuration issue, not a code bug. The system is working as designed - it requires API keys for all providers in use.