AI System Design

Model-Aware Architecture

The model is a capability inside a distributed system, not the whole system. Design identity, data flow, retrieval, orchestration, observability, and fallback around it.

Zero Trust AI Access

Prompts, retrieved context, embeddings, memory, tool calls, generated outputs, and logs are treated as protected assets. Apply continuous identity, device, location, risk, data-sensitivity, and action-context checks before model access, retrieval, memory use, or tool execution — using established Zero Trust and Conditional Access patterns from your identity provider (e.g., Microsoft Entra ID, Okta, AWS IAM Identity Center).

Trade-Off Driven

Every design choice should explain the selected option, the rejected option, and the cost, latency, quality, privacy, or operational impact accepted.

Least-Privilege Agent Authority

Agents should never inherit broad human or platform permissions. Tool access must be scoped, authorized, logged, and revoked when risk changes.

Grounded and Evaluated Outputs

Use retrieval, citations, validation, test sets, red-team scenarios, and human review where AI output must be trusted or acted upon.

Operational Resilience

Design the AI platform as an operations-grade system with capacity controls, cost governance, lifecycle management, model rollback, and measurable service health.

Threat-Informed Design

Map design controls to the full adversarial ML attack taxonomy (NIST AI 100-2e2025): evasion, data poisoning, backdoor/trojan, model extraction, membership inference, and prompt injection — aligned with MITRE ATLAS techniques and OWASP LLM Top 10 application-layer controls.

Graceful Degradation

When model, vector index, tool API, or guardrail services fail, the system should degrade safely rather than fail open or take uncontrolled action.

Incomplete Defense Acknowledgment

Prompt injection, particularly indirect injection via external content, has no known complete defense as of 2025. Defense-in-depth reduces risk but cannot eliminate it for high-assurance contexts requiring human approval gates.

AI RMF Operating Functions

Use Govern, Map, Measure, and Manage as the lifecycle operating model for AI risk decisions, ownership, assessment, monitoring, and treatment.

Trustworthiness Characteristics

Design for valid and reliable behavior, safety, security and resilience, accountability and transparency, explainability and interpretability, privacy enhancement, and harmful-bias management.

NIST AI 600-1 GenAI Risk Taxonomy

Screen the design for generative-AI risks unique to, or exacerbated by, model generation, retrieval, tool use, agentic execution, and synthetic content creation. Twelve risk areas from the March 2024 Generative AI Profile.

EU AI Act Obligations

Classify the system under the EU AI Act risk tiers. Prohibited practices must be blocked at design time. High-risk AI systems require conformity assessment, technical documentation, human oversight, bias testing, accuracy and robustness standards, and registration in the EU database before market placement.

Risk-to-Control Traceability

Each high-risk AI behavior should trace to questionnaire evidence, design factor, security EPIC, pattern selection, trade-off rationale, test case, and monitoring signal.

Governance Evidence

Capture model card inputs, data lineage, prompt versions, retrieval source policy, evaluation results, red-team findings, approval decisions, incident records, and exception handling.

Decision Gates

Require explicit review before production release, model/provider change, new tool enablement, new sensitive data class, increased autonomy, or expanded user population.

Sector Regulations

Layer sector-specific frameworks — HIPAA, PCI DSS, SOX, GDPR, DORA (financial AI), MiFID II — over the AI-specific governance baseline. Requirements may restrict model hosting, data residency, explainability, and human oversight depth.

Model Inference

• Provider, model class, region, latency, context window, fallback, and cost controls.
• Trade-off: quality and reasoning depth versus latency, cost, and data handling restrictions.

Retrieval and Grounding

• Source ingestion, chunking, embeddings, vector index, source ranking, citations, and authorization.
• Trade-off: grounded answers versus retrieval latency and governance complexity.
• Authorization must occur at retrieval time — never only post-generation.

Agent Orchestration

• Planner, state machine, tool registry, approval gates, retry policy, and compensating actions.
• Trade-off: automation speed versus control, auditability, and safety.

Tool Execution

• Read-only, write, privileged, financial, customer-facing, and cloud-admin action classes.
• Trade-off: operational leverage versus blast-radius and misuse risk.

Memory and Privacy

• Session memory, approved long-term memory, tenant memory, retention, deletion, and consent.
• Trade-off: personalization versus privacy, retention, and leakage exposure.

Evaluation and Assurance

• Golden tests, red-team scenarios, prompt/version regression, retrieval quality, and guardrail metrics.
• Trade-off: confidence and repeatability versus evaluation cost and release friction.

Layer 1: Operations, Resilience, and Scale

• Capacity planning, quotas, request scheduling, autoscaling, cost controls, high availability, and degraded modes.
• Design triggers: high concurrent requests, strict latency targets, provider rate limits, GPU scarcity, or business-critical workflows.

Layer 2: Data Integrity and Vector Isolation

• Input validation, source provenance, chunking governance, embedding drift control, vector-store authorization, and tenant isolation.
• Design triggers: enterprise RAG, regulated records, frequent knowledge updates, or per-user document authorization.

Layer 3: Guarded Inference Pipeline

• User input capture, prompt policy, semantic cache, retrieval routing, model inference, output validation, and redaction.
• Design triggers: sensitive prompts, customer-visible outputs, hallucination risk, or response quality requirements.

Layer 4: Securing the Agentic Baseline

• Structured playbooks, tool registry, policy engine, execution sandbox, approval gates, audit trail, and kill switch.
• Design triggers: agents, external tools, write actions, cloud operations, customer-impacting tasks, or multi-step workflows.

Observe-Think-Act Threat Surface

• Observe: memory/context, retrieved documents, and external inputs. Think: prompt planning and reasoning chain. Act: tool calls, APIs, code, and workflow execution.
• Control each stage with validation, authorization, monitoring, and containment. No single defense is sufficient for indirect injection.

Diagnostic Playbook

• Symptoms: peak-traffic crashes, retrieval threat exposure, instruction override, fine-tune drift, hallucination, context saturation, and overfitting.
• Responses: inference scaling, retrieval isolation, prompt/firewall policy, retraining governance, grounding, progressive disclosure, and regularization.

LLM Fundamentals

Account for tokenization, context window, temperature, structured output, KV cache, model size, and model behavior constraints before capacity and UX targets are finalized.

Prompt Engineering

Design prompt templates, system instructions, output parsers, prompt chaining, edge-case handling, multilingual behavior, and prompt injection defenses as versioned assets. No perfect injection defense exists — design for detection, not prevention alone.

Retrieval-Augmented Generation

Evaluate chunking, embeddings, hybrid search, reranking, metadata filtering, source attribution, GraphRAG, freshness, access control, and multi-hop retrieval. Authorization at retrieval time is mandatory — not post-generation.

AI Agents and Agentic Systems

Design tool definitions, Model Context Protocol (MCP) boundaries, ReAct or plan-and-execute flows, memory, agent loop termination, orchestration, recovery, and human approval.

Fine-Tuning and Model Adaptation

Choose prompt engineering, RAG, LoRA, QLoRA, full fine-tuning, distillation, or (rarely, with research-scale infrastructure) continual pre-training based on data stability, hardware, latency, cost, privacy, and forgetting risk.

Vector Databases and Embeddings

Plan embedding model choice, dimensionality, index strategy, metadata, multi-tenancy, drift, quantization, and search-quality benchmarking. Embeddings are derived sensitive data where source data is sensitive.

LLMOps and Production AI

Operationalize model serving, prompt versioning, model rollback, feature flags, rate limiting, token streaming, cost controls, model routing, and provider fallback.

Evaluation and Testing

Use golden datasets, regression tests, red teaming, hallucination detection, factual consistency checks, RAG metrics, agent evaluation, and continuous evaluation.

Safety, Ethics, and Responsible AI

Address hallucination, bias, privacy, explainability, misuse, copyright, audit trails, incident response, EU AI Act classification, high-risk AI conformity, and user appeal workflows.

Multimodal and Infrastructure Scale

Extend architecture for images, tables, audio, video, multi-modal RAG, GPUs, batching, speculative decoding, quantization, request priority, and edge or on-device deployment.