AI engineering workflows

Local analysis first

Prompt analysis produces structural findings, ambiguity and conflict checks, security findings, and evaluation cases. Prompt histories preserve versions and reasons for changes. These local reports do not need a model key.

Live evaluations

Provider-backed workflows can execute cases, retain outputs and errors, compare models, and inspect behavioral differences. You choose the endpoint, model, and environment variable containing credentials. Consult fehm --help for the prompt subcommands and configuration flags.

Promotion with held-out cases

Prompt optimization uses bounded candidate strategies and a separate held-out promotion gate. Evaluator scores are model-produced evidence, not proof of correctness. Small test suites can miss regressions.

Graph-backed claims

fehm ai-evaluations
fehm agent analytics

Agent records and evaluation graphs connect prompts, tests, tools, and outcomes. Claim verification checks concrete repository statements and leaves unsupported absolutes unverified.

Control the boundary

Provider calls may transmit prompts and selected source context. Review data handling, budget, latency, credentials, and failure behavior before a live run. Start with a small evaluation to check the provider configuration.