The Problem
Most tutorials show "connect OpenAI API and call chat completion." In production you need:
- Different models for different tasks (GPT-4o for resume analysis, Haiku for summarization)
- Hard budget cap — so a junior HR doesn't spend $500 overnight
- Full audit log of every call with cost tracking
- Fallback when a provider is down
- Ability to switch to self-hosted (Ollama, vLLM) without code changes
Architecture
AiProvider — API Abstraction
class AiProvider
PROVIDERS = {
openai: AiProviders::OpenAi,
anthropic: AiProviders::Anthropic,
openrouter: AiProviders::OpenRouter,
ollama: AiProviders::Ollama,
custom: AiProviders::Custom,
}.freeze
def self.for(task_name)
config = AiTaskConfig.find_by(task: task_name) || default_config
PROVIDERS[config.provider.to_sym].new(config)
end
end
Each provider implements one method #chat(messages:, **opts) returning a standard AiResponse.
Per-task Configuration
In /settings/ai, HR picks the model for each task. Heavy reasoning tasks get GPT-4o, simple summarization gets Haiku.
AiBudgetGuard
Checked before every RunAiTaskJob. Cached for 60 seconds. Hard cap at 2x monthly budget.
Audit Log
Every AI call creates an AiRun record with provider, model, tokens, cost, duration, status. Real-time cost dashboard in admin.
Key Takeaways
- Provider abstraction pays off from day one — switching models takes 30 seconds in the UI
- Budget guard is mandatory — without it, the first curious HR manager burns the monthly budget in an hour
- Audit log is a compliance necessity (GDPR), not a luxury
- Self-hosted fallback (Ollama) saves you during OpenAI downtime
Comments (0)