AIT-APP-01
Generative AI workflow design
Designs a bounded AI workflow aligned with a defined professional problem and output contract.
Loading your workspace…
Loading your workspace…
AI & TECHNOLOGY · COMPLETE DRAFT · READY FOR REVIEW
A project-centered course for designing, implementing, and evaluating a bounded generative-AI workflow for a real professional task.
GOVERNANCE GATE
This version exposes the proposed sources, competencies, instruction, evidence requirements, and rubric for accountable review. It cannot be enrolled in, completed, or used to issue a credential until an authorized reviewer approves a final immutable version and the program is separately published.
Status: Complete draft awaiting accountable instructional review
Draft version: 1.0.0-draft.1
Estimated learning time: 190 minutes
Program level: Course
Evidence activities: 3
Credential pathway: Proposed Certificate of Completion after review, publication, learner evidence approval, and separate administrator issuance.
LEARN → APPLY → DEMONSTRATE → REVIEW → VERIFY
AIT-APP-01
Designs a bounded AI workflow aligned with a defined professional problem and output contract.
AIT-APP-02
Builds representative tests and interprets quality, reliability, latency, and cost evidence.
AIT-APP-03
Defines data controls, human approval, monitoring, and failure response for deployment.
CONTINUING PROFESSIONAL CASE
A services team wants generative AI to classify incoming requests, draft a response, and route urgent cases. Early prototypes look impressive but mishandle ambiguous requests and expose account details in logs.
Constraint: The first release must remain assistive, use a limited budget, and provide measurable evidence before broader automation.
Learner task: Design a bounded prototype, evaluation set, operating controls, and evidence-based release recommendation.
FIVE SOURCE-GROUNDED LESSONS
LESSON 1
Purpose: Define the problem before selecting a model.
A useful AI project begins with current workflow evidence, a defined user, measurable success, and explicit non-goals. Model capability is only one design constraint.
Professional example: The prototype drafts but never sends responses and routes uncertain cases to a person.
Evidence connection: Evaluation guidance recommends defining the task objective and what good performance means before optimizing.
Write the triage workflow's user, outcome, non-goals, and release criteria.
What should come first?
Expected: Define the user, problem, success criteria, and non-goals. — Problem and evidence should govern model choices.
LESSON 2
Purpose: Make model behavior testable and integrable.
A workflow should validate inputs, constrain output shape, handle missing information, and distinguish generated suggestions from authoritative system state.
Professional example: The classifier returns a validated category, urgency, evidence excerpt, and uncertainty flag.
Evidence connection: Current model APIs support structured output and tool contracts, while evaluation remains necessary for semantic quality.
Define the triage input and output schema plus behavior for missing account context.
What does structured output guarantee?
Expected: A predictable shape when validated, not semantic correctness. — Structure improves integration but does not prove quality.
LESSON 3
Purpose: Test the workflow against real variation and failure.
An evaluation set should represent common cases, edge cases, ambiguous inputs, and high-cost failures. Metrics should connect to the professional decision, not merely generic model preference.
Professional example: The set includes routine, ambiguous, multilingual, adversarial, and urgent customer requests.
Evidence connection: OpenAI evaluation guidance emphasizes task-specific datasets, criteria, and continuous evaluation.
Create a stratified triage test set and define per-category acceptance thresholds.
Which evaluation is strongest?
Expected: A representative set with edge cases and explicit acceptance criteria. — Representative evidence supports defensible release decisions.
LESSON 4
Purpose: Improve the system from observed evidence.
Teams should classify failures, trace them to data, instructions, model choice, or workflow design, then change one meaningful variable and rerun the evaluation.
Professional example: Ambiguous requests trigger clarification instead of forced classification.
Evidence connection: Evaluation practice treats improvement as an empirical cycle of measurement, analysis, change, and regression testing.
Analyze three triage failures and select a design change with a regression test.
What is the best response to a recurrent failure?
Expected: Classify the cause, revise the relevant layer, and rerun regression tests. — Controlled iteration connects changes to evidence.
LESSON 5
Purpose: Turn a prototype into a governed service.
Production readiness includes data handling, rate and cost limits, monitoring, incident ownership, human approval, version records, and a tested rollback path.
Professional example: The system redacts identifiers, caps spend, records versions, and routes all low-confidence cases to staff.
Evidence connection: NIST frames generative-AI risk management across design, deployment, monitoring, and incident response.
Write the triage workflow's privacy, monitoring, escalation, and rollback runbook.
What is required beyond offline quality?
Expected: Operational controls for data, cost, monitoring, human review, and rollback. — Operational evidence and controls are separate from model quality.
ASSESSED APPLICATION
PROPOSED REVIEW RUBRIC
Problem framing and operational boundary is explicit, workable, and supported by relevant evidence.
Competency: AIT-APP-01
Technical design and implementation rationale is explicit, workable, and supported by relevant evidence.
Competency: AIT-APP-02
Evaluation design and preserved evidence is explicit, workable, and supported by relevant evidence.
Competency: AIT-APP-03
Risk controls and accountable governance is explicit, workable, and supported by relevant evidence.
Competency: AIT-APP-03
Failure analysis, revision, and professional judgment is explicit, workable, and supported by relevant evidence.
Competency: AIT-APP-03
PRIMARY SOURCE BASE
OpenAI · OpenAI API Documentation · Current documentation accessed August 2026
Provides current primary guidance for making model requests, handling outputs, and adding tools.
OpenAI · OpenAI API Documentation · Current documentation accessed August 2026
Provides current primary guidance for defining objectives, datasets, metrics, and continuous evaluation of AI applications.
National Institute of Standards and Technology · NIST AI 600-1 · 2024
Provides a cross-sector framework for identifying, measuring, and managing generative-AI risks across the lifecycle.