An AI demo is a prompt and a response.

A production AI feature is a trust boundary.

It reads data, makes suggestions, may call tools, affects users, costs money, and fails in ways ordinary deterministic code does not.

That changes the engineering.

Say what the feature does—and does not do

“AI assistant” is too broad.

Define a bounded capability:

• drafts a reply from selected conversation history;
• classifies an incoming request into approved categories;
• extracts named fields from one document type;
• summarizes a report for human review;
• suggests next actions without executing them.

Then state the limitation visibly.

Do not market probabilistic output as guaranteed accuracy.

Do not let the UI imply authority the system does not have.

Treat retrieved text as untrusted input

Documents, webpages, emails, and user messages can contain instructions aimed at the model.

That is prompt injection.

Retrieval does not turn text into trusted policy.

I separate:

• system policy;
• developer instructions;
• user intent;
• retrieved evidence;
• tool results.

The application decides which tools exist and what arguments are allowed.

A document should not be able to grant itself permission.

Minimize and scope data

Before retrieval or generation:

• verify actor and tenant;
• filter records server-side;
• remove unnecessary fields;
• enforce document ownership;
• limit time range and count;
• avoid secrets and hidden metadata;
• record the source IDs used.

The model should receive the smallest context that can complete the task.

Large context is not automatically better context.

Constrain tools like public APIs

If the model can send email, update records, issue refunds, or run queries, every tool is a protected server entry point.

Each call needs:

• schema validation;
• authorization;
• business-rule checks;
• rate limits;
• idempotency where needed;
• safe error handling;
• audit events;
• bounded output.

Never trust a tool call because the model produced valid JSON.

Valid shape is not valid permission.

Validate output before use

For structured output:

• parse against a schema;
• reject unknown or impossible values;
• enforce lengths;
• normalize identifiers;
• verify referenced records;
• escape or sanitize display;
• keep generated text out of executable contexts.

For factual summaries, preserve links to the underlying records so a human can inspect the source.

Do not add decorative citations the system cannot verify.

Put human control at the right point

High-impact action should not hide behind automatic language.

Useful review includes:

• original source;
• generated proposal;
• uncertainty or missing data;
• edit;
• approve;
• reject;
• reason where it helps operations.

The reviewer must understand that they own the decision.

If users approve everything because the interface makes rejection painful, the human control is ceremonial.

Design failure as a normal state

Expect:

• timeout;
• rate limit;
• provider outage;
• malformed output;
• safety refusal;
• context too large;
• stale retrieval;
• partial tool completion;
• duplicate retry.

The UI should explain:

• what happened;
• whether anything changed;
• whether retry is safe;
• how to continue manually.

Silently switching models or prompts can change behavior. Version and observe those changes.

Log decisions without collecting everything

An audit record may need:

• feature version;
• model and prompt version;
• actor and tenant IDs;
• source record IDs;
• tool names;
• outcome;
• approval state;
• error code;
• request trace.

It may not need the full private prompt or every raw document.

Retention should have a purpose and an owner.

Evaluate the workflow, not one answer

Offline examples are useful.

Production evaluation also needs:

• completion rate;
• human edit distance;
• rejection rate;
• harmful or invalid output;
• latency;
• cost;
• fallback use;
• user outcome;
• differences across languages and user groups.

One impressive response is not quality evidence.

Keep the rollout reversible

Use:

• feature flags;
• limited cohorts;
• usage caps;
• approval requirements;
• model/prompt versioning;
• kill switch;
• manual fallback;
• rollback notes.

The first production release should make learning cheap and failure contained.

My standard

I consider an AI feature ready when:

• its capability is narrow and honest;
• data access is tenant-safe;
• retrieved content is untrusted;
• tools enforce permission;
• output is validated;
• humans control high-impact decisions;
• failure is recoverable;
• behavior is observable;
• rollout can stop.

That is the engineering layer behind responsible AI.

For deciding whether AI belongs in the workflow at all, use my AI automation opportunity checklist. For tenant data, read secure multi-tenant SaaS foundations.

If your SaaS AI feature is moving from demo to production, send me its data source, action, and worst credible error.