ai

Call AI models — generate, stream, and structured output with provider fallback

Overview

The ai module wraps Vercel AI SDK behind the toolkit's adapter pattern. It provides text generation, streaming, and structured output with automatic provider detection, fallback, cost tracking, and rate limiting.

Peer dependencies: ai, plus one provider SDK (@ai-sdk/groq, @openrouter/ai-sdk-provider, @ai-sdk/anthropic, or @ai-sdk/openai)

npm install ai @ai-sdk/groq

yarn add ai @ai-sdk/groq

pnpm add ai @ai-sdk/groq

Quick Start

import { createAI } from '@jamaalbuilds/ai-toolkit/ai';

const ai = createAI(); // auto-detects from GROQ_API_KEY
const result = await ai.generate('Explain TypeScript generics');
console.log(result.text);

API Reference

createAI(config?)

Create an AI client with provider fallback and cost tracking. Auto-detects provider from env vars: GROQ_API_KEY → OPENROUTER_API_KEY → ANTHROPIC_API_KEY → OPENAI_API_KEY.

function createAI(config?: AIConfig): AIClient

Returns: AIClient with generate(), stream(), and structured() methods.

const ai = createAI({ provider: 'groq', fallbackProvider: 'openrouter' });
const ai = createAI({ maxRequestsPerMinute: 30, maxTokensPerDay: 100000 });

ai.generate(prompt, options?)

Generate text from a prompt.

ai.generate(prompt: string, options?: GenerateOptions): Promise<GenerateResult>

const result = await ai.generate('What is RAG?', {
  system: 'You are a helpful AI teacher.',
  temperature: 0.7,
  maxTokens: 500,
});

console.log(result.text);          // generated text
console.log(result.usage);         // { inputTokens, outputTokens, totalTokens }
console.log(result.cost.totalCost); // estimated cost in USD
console.log(result.latencyMs);     // response time
console.log(result.usedFallback);  // whether fallback provider was used

ai.stream(prompt, options?)

Stream text from a prompt. Returns an async iterable for real-time output.

ai.stream(prompt: string, options?: StreamOptions): Promise<StreamResult>

Additional options beyond GenerateOptions:

const stream = await ai.stream('Write a haiku about TypeScript');

for await (const chunk of stream.textStream) {
  process.stdout.write(chunk);
}

const fullText = await stream.text;    // complete text after streaming
const usage = await stream.usage;       // token usage after streaming

ai.structured(prompt, options)

Generate structured output matching a Zod schema.

ai.structured<T>(prompt: string, options: StructuredOptions<T>): Promise<StructuredResult<T>>

import { z } from 'zod';

const result = await ai.structured('John is a 30-year-old software engineer', {
  schema: z.object({
    name: z.string(),
    age: z.number(),
    occupation: z.string(),
  }),
  schemaName: 'Person',
});

console.log(result.object); // { name: 'John', age: 30, occupation: 'software engineer' }
// result.object is fully typed as { name: string; age: number; occupation: string }

Provider Fallback

When the primary provider fails (rate limit, outage), the client automatically retries with the fallback provider:

const ai = createAI({
  provider: 'groq',             // primary: fast + free
  fallbackProvider: 'openrouter', // fallback: access to many models
});

const result = await ai.generate('Hello');
console.log(result.usedFallback); // true if Groq failed

Rate Limiting

Built-in rate limiting prevents exceeding provider quotas:

const ai = createAI({
  maxRequestsPerMinute: 30,
  maxTokensPerDay: 100_000,
});

Throws LLMError with code: 'LLM_RATE_LIMITED' when limits are exceeded.

Types

• AIClient — client interface with generate/stream/structured
• AIConfig — configuration options
• AIProvider — 'groq' | 'openrouter' | 'anthropic' | 'openai'
• GenerateResult — text, model, provider, usage, cost, latencyMs
• StreamResult — textStream, text promise, usage promise
• StructuredResult<T> — typed object, model, provider, usage, cost
• TokenUsage — inputTokens, outputTokens, totalTokens
• CostEstimate — inputCost, outputCost, totalCost, currency