End-to-End ML Lifecycle on AWS

Machine learning projects rarely fail because a model cannot be trained. They fail because nobody decided which stage of the lifecycle owns which service, because a notebook prototype was shipped as a permanent endpoint, or because nobody noticed the input data changed six months after launch. The AIF-C01 exam is built around exactly that gap: Task 1.3 asks you to place data collection, EDA, pre-processing, feature engineering, training, hyperparameter tuning, evaluation, deployment and monitoring against AWS services, and Task 1.1 asks you to distinguish batch, real-time, asynchronous and serverless inferencing.

By the end of this lesson you will be able to:

• place each of the nine lifecycle stages against the SageMaker feature and the non-SageMaker service that supports it;
• apply AWS's three-question selection framework (build vs. customize vs. call an API, modality, control) before writing any code;
• configure the Feature Store online and offline stores and explain why a fraud model needs both;
• size a serverless endpoint, compute batch transform concurrency, and know the exact ceiling of each of the four inference options;
• build the drift → CloudWatch → EventBridge → Pipelines retrain loop and explain which monitor detects which drift;
• defend a comparative verdict: SageMaker vs. building on EC2, vs. non-AWS platforms, vs. a manual process;
• spot the maintenance and sunset traps (Ground Truth, Model Monitor, Clarify, Studio Lab, Mechanical Turk) that appear in 2026 exam items;
• answer ten exam-style questions written in the official practice-question voice.

---

1. The nine stages of the ML lifecycle on AWS

1.1 Stage-by-stage service map

This table is the single most examinable artifact of Task 1.3. The left column is the exam guide's own vocabulary; the middle columns are the services you must be able to name under pressure.

1.2 The MLOps concepts the exam tests

Task 1.3 does not stop at stages. It also names seven MLOps fundamentals: each one maps to a concrete AWS mechanism:

1.3 The lifecycle is a loop, not a line

The arrow that returns from Monitoring to Training is the whole point of MLOps. A team that deploys and never returns to stage 5 has a project, not a system, and that is exactly the "technical debt" and "production readiness" language the exam uses.

Order the lifecycle before you memorize the services:

---

2. Choosing the AWS path: the three-question framework

Before you pick a service, AWS wants you to answer three questions in order. Skipping to "SageMaker" is the most common architectural anti-pattern in the exam.

2.1 Question 1: call an API, customize a foundation model, or build your own?

2.2 Question 2: what is the modality of the data?

2.3 Question 3: how much control do you need?

AWS documents a control ladder that climbs from no-code to full-code:

AWS also warns in its own guidance that Studio is the wrong starting point for a team with no ML experience: start with Canvas.

2.4 Reference architecture: one account, end to end

📚 Did you know? SageMaker Pipelines are literally a JSON DAG. The documented step types are Processing, Training, Tuning, AutoML, RegisterModel, DeployModel, Transform, Condition, Callback, Lambda, ClarifyCheck, QualityCheck, EMR, Notebook Job and Fail, and step outputs feed the inputs of downstream steps, which is what makes the loop in the diagram above automatic.

---

3. Data ingestion, storage and the Feature Store

3.1 Where the data comes from

3.2 Feature Store: online vs. offline

3.3 Online store tiers and write semantics

AWS documents two online tiers:

• Standard: the default online store configuration;
• InMemory: backed by Amazon ElastiCache (Redis OSS) for the fastest reads. It is online-only (no online↔offline replication) and it does not support customer-managed KMS keys.

On the write path, PutRecord.TargetStores accepts {OnlineStore, OfflineStore} with a maximum of 2 entries, so a single call can update both stores, but only if a group has both enabled.

Example 5, feature freshness with real numbers. A fraud model calls GetRecord and gets an answer in single-digit milliseconds. The same feature group's offline store keeps every historical version as Parquet in S3, which Athena can query for a point-in-time correct training set. A PutRecord issued at 14:00 reaches S3 no later than 14:15 (15-minute buffering). Training therefore sees a slightly delayed snapshot while inference sees the newest row, which is precisely why the exam asks for both stores enabled on one feature group.

📚 Did you know? The Feature Store identity is not just a primary key: a record is uniquely identified by the feature identifier plus the event time, and the offline store additionally records write_time, api_invocation_time and an is_deleted flag. That is what makes point-in-time joins possible: a training query at 09:00 can only see features whose event time is before 09:00, even if they were written later.

---

4. Data processing, EDA and feature engineering

4.1 Which service processes which workload

4.2 The Processing job contract

A SageMaker Processing job has a four-step contract that the exam likes to test:

That "compute is released" property is why Processing jobs beat a permanently attached notebook instance for anything scheduled.

4.3 Where the no-code tools live now

Two migrations matter for the 2026 exam:

• Data Wrangler is integrated into SageMaker Canvas; it is no longer a separate Studio sidebar experience;
• Autopilot's UI moved to Canvas on 30 Nov 2023: the AutoML API remains documented and callable, but the visual AutoML experience is Canvas.

📚 Did you know? Amazon Athena charges per query and per gigabyte scanned, so the cheapest EDA is a partitioned Parquet table plus a WHERE dt = '…' filter. A full-table scan of a 1 TB CSV costs roughly 100× more than the same query over partitioned Parquet: the exam does not ask for the price, but it does ask whether Athena is "serverless SQL over S3".

---

5. Training, hyperparameter tuning and orchestration

5.1 Training jobs and the cost ceiling

5.2 Ways to train without writing the loop yourself

5.3 Pipelines: the orchestration backbone

A SageMaker Pipeline is a JSON DAG whose steps exchange outputs as inputs. The documented step types are: Processing, Training, Tuning, AutoML, Register/Deploy model, Transform, Condition, Callback, Lambda, ClarifyCheck, QualityCheck, EMR, Notebook Job and Fail.

5.4 Model Registry

The Model Registry is the production-grade answer to "which model is live?". It stores versioned Model Package Groups, and each Model Package carries metrics, lineage, a staging stage, an approval status, and hooks for CI/CD deployment and sharing across accounts. Anything that is not in the Registry is, in exam terms, an untracked experiment.

---

6. Evaluation, documentation and governance

6.1 Amazon SageMaker Clarify: three different analyses

Clarify writes JSON bias metrics, global attributions and a visual report to S3, and it can run against a live endpoint or against a temporary shadow endpoint that Clarify creates and then deletes.

6.2 Model Cards and Model Dashboard

If the exam says "the model owner must record intended uses, risk rating, training details and evaluation results", the answer is Model Cards: not the Dashboard (which aggregates) and not Model Monitor (which only tracks drift).

---

7. Deployment: the four inference options

This is the central service-selection topic of the whole lesson. SageMaker AI documents exactly four inference options, and each one has a hard ceiling you must be able to recall.

7.1 Batch vs. real-time vs. asynchronous vs. serverless

7.2 The numeric ceilings in one place

Check the ceilings before you pick an option: the payload decides, not the latency preference:

7.3 Real-time endpoints in detail

• Persistent HTTPS endpoint invoked with InvokeEndpoint; response streaming uses InvokeEndpointWithResponseStream.
• Bidirectional HTTP/2 on port 8443, sessions capped at 30 minutes.
• Several models can share one endpoint through inference components: the exam's answer when it asks to host multiple models without paying for multiple endpoints.

7.4 Serverless Inference in detail

You choose memory; AWS assigns vCPUs proportionally. Cold-start time is driven by model size + model download + container start, which is why the memory must be at least the size of your model.

Serverless has documented exclusions: no GPUs, no AWS Marketplace packages, no private registries, no multi-model endpoints, no VPC configuration, no data capture, no Model Monitor, and no inference pipelines. And the conversion is one-way: converting an existing real-time endpoint to serverless fails with a ValidationError, while the reverse conversion is allowed and one-way.

7.5 Asynchronous Inference in detail

InvokeEndpointAsync takes its input from S3, writes output to S3, and sends a completion notification. Requests queue while capacity is zero, and autoscaling can scale the instance count to 0; that combination (queue + scale to zero) is what makes async the right answer for large, slow payloads.

7.6 Batch transform in detail

• SplitType=Line splits the input file into mini-batches;
• BatchStrategy=MultiRecord|SingleRecord with MaxPayloadInMB;
• CSV with embedded newlines is not supported;
• MaxPayloadInMB=0 means chunked streaming (not for built-in algorithms);
• default output name is input1.csv.out;
• the ideal MaxConcurrentTransforms equals the number of parallel workers you want.

7.7 Choosing among the four

Example 1: option choice by payload. A fraud API receives 3 KB payloads and must answer in under 200 ms → real-time (ceilings 25 MB / 60 s). Later the same model is asked to score 800 MB videos that take 10 minutes → that exceeds the 25 MB real-time ceiling but fits asynchronous (≤ 1 GB, ≤ 60 min), so you switch options and keep scale-to-zero.

Example 2: serverless sizing. A 3.5 GB model needs MemorySizeInMB=4096 (memory must be ≥ model size; the allowed range is 1024…6144). Set MaxConcurrency=50 (allowed 1 to 200) with ProvisionedConcurrency=10 (must be ≤ MaxConcurrency) to pre-warm part of the traffic, and keep the container image ≤ 10 GB.

Example 3: batch concurrency math. With MaxPayloadInMB=25, the maximum legal MaxConcurrentTransforms is 4, because 4 × 25 = 100 ≤ 100 MB. Setting 5 would break the documented rule (5 × 25 = 125 > 100).

---

8. Monitoring, drift and the retraining loop

8.1 The four monitor types

All monitors emit metrics and violations to Amazon CloudWatch, which is the exam's answer whenever it mentions "raise an alarm on thresholds". Note also that the prebuilt monitoring container supports tabular data only: NLP and computer-vision models need a bring-your-own container.

8.2 From alarm to retrained model

Example 6: the drift → retrain loop with real numbers. A Deequ baseline is computed from the training data. A scheduled monitoring job compares live traffic against it. Feature-attribution monitoring raises an automatic alarm when NDCG drops below 0.90. A bias example uses a DPPL range of (−0.1, 0.1) evaluated over 2 days with bootstrap confidence intervals. The alarm goes to CloudWatch, CloudWatch triggers an EventBridge rule, and EventBridge starts a SageMaker Pipelines retrain run whose new version lands in the Model Registry as PendingManualApproval.

8.3 What AWS now recommends instead

Because Model Monitor and Clarify are closed to new customers, AWS's own availability-change documentation points at a replacement stack:

📚 Did you know? Experiments Classic is available only in Studio Classic, and AWS now recommends managed MLflow 3.0 with Studio Apps: experiment artifacts are stored in your S3 bucket, so the tracking data outlives the notebook. MLflow is also the backbone of the recommended replacement for Model Monitor.

---

9. Comparative verdict: SageMaker vs. the alternatives

9.1 SageMaker vs. individual services

AWS's own decision criteria are: infrastructure management, compute commitments, pricing preference, model-architecture support, and control over engine, configuration and infrastructure.

9.2 Amazon Bedrock vs. Amazon SageMaker: the official line

9.3 Streaming inference architecture

The pattern AWS documents in its architecture blogs:

---

10. Availability, maintenance and sunset traps

The 2026 exam actively tests availability. Answering correctly means knowing what a maintenance notice means and what a sunset means.

10.1 Availability snapshot (SageMaker, 2026)

The standard maintenance wording is: "Customers can't on-board… AWS will continue to operate and support these services… but won't enhance or add functionality." Existing customers keep access; there are simply no new features.

10.2 SageMaker feature map and status

📚 Did you know? Studio Lab, the free JupyterLab 4 environment that required no AWS account and no credit card, had published limits of just 12 GB RAM and 15 GB of storage, and it is now closed to new customers with SageMaker Studio as the replacement. That is why "use Studio Lab for free" is a distractor in 2026 material.

10.3 2025 to 2026 Updates

Between December 2024 and October 2026 the SageMaker ecosystem was renamed, trimmed and re-pointed several times. Every row below comes from an AWS announcement, an AWS availability-change page or the AWS certification change history: nothing here is third-party, and dates AWS itself does not publish are deliberately omitted (see section 13).

Two vocabulary facts complete the picture:

• Maintenance = no new customers, no new features, still supported · sunset = planned end of operations, typically announced about a year ahead · full shutdown = removed from the AWS portfolio. AWS's Service Lifecycle reference defines all three, and every row above sits in the first two states.
• Exam-guide updates land on the test about one month after publication, so anything AWS published on v1.1 (30 Apr 2026) is already fair game: including SageMaker JumpStart, Bedrock AgentCore, Kiro and Strands Agents on the in-scope list.

📚 Did you know? The Kendra availability page does not say "deprecated": it redirects new search applications to Amazon Bedrock Managed Knowledge Base, whose Smart Parsing and Agentic Retrieval API are documented as the Kendra replacement. "Kendra was shut down" is therefore a distractor: Kendra is in maintenance, which under AWS's own vocabulary means it keeps running for existing customers.

---

11. Cost, free tier and the numbers you can safely memorize

11.1 SageMaker free tier (each month, for the first 2 months)

11.2 Worked cost pattern

AWS's own worked example totals $2.38: 2 hours of ml.m4.4xlarge at $0.96 each plus 2 hours of ml.m5.xlarge at $0.23 each, with built-in Debugger rules billed at $0. Learn the pattern (instance-hours × rate + job surcharges), never the rate: ml.* prices are region- and date-dependent.

11.3 Quotas

Endpoint counts, pipeline quotas, concurrent-job limits and serverless concurrency live in Service Quotas. Never guess a numeric quota in an exam answer or a design review: look it up.

---

12. Worked AWS examples with numbers

Example 1: option choice by payload (section 7.7). Fraud API: 3 KB payloads, < 200 ms → real-time (25 MB / 60 s ceilings). Same model scoring 800 MB videos that take 10 minutes → exceeds 25 MB but fits asynchronous (≤ 1 GB, ≤ 60 min). Switch options; keep scale-to-zero.

Example 2: serverless sizing (section 7.4). Model 3.5 GB → MemorySizeInMB=4096; MaxConcurrency=50 (range 1 to 200); ProvisionedConcurrency=10 (≤ 50) to pre-warm; container image ≤ 10 GB.

Example 3: batch concurrency math (section 7.6). MaxPayloadInMB=25 → maximum MaxConcurrentTransforms=4 because 4 × 25 = 100 ≤ 100 MB. Five workers would violate the documented rule.

Example 4: free-tier budget. 250 notebook hours ≈ 8.3 instance-days; 50 training hours ≈ six 8-hour runs; 125 real-time hours ≈ 4.2 days of hosting; 150,000 serverless seconds ≈ 41.7 billable hours. Exceeding 250 notebook hours in month 1 starts metered billing.

Example 5: feature freshness (section 3.2). Online GetRecord answers in single-digit milliseconds; the offline store keeps every version as Parquet in S3 (Athena-queryable) for point-in-time joins; a 14:00 PutRecord reaches S3 by 14:15 at the latest.

Example 6: drift → retrain loop (section 8.2). Deequ baseline from training data; scheduled monitoring of live traffic; attribution drift auto-alarms below NDCG 0.90; bias example uses DPPL ∈ (−0.1, 0.1) over 2 days with bootstrap confidence intervals → CloudWatch → EventBridge → Pipelines retrain.

Example 7: training cost ceiling (section 5.1). A training job with the default MaxRuntimeInSeconds stops after 1 day; the hard maximum is 28 days, and a job cannot exceed 30 days total. On stop SageMaker sends SIGTERM and waits 120 seconds for your checkpoint code to finish.

---

13. Conflicting, unverified and non-examinable facts

Good exam candidates separate what AWS documents from what AWS marketplaces. Keep this list out of your memorization set except where flagged.

Primary sources for this lesson: the AIF-C01 Exam Guide; the SageMaker developer guide pages deploy-model-options, hosting-faqs, serverless-endpoints, batch-transform, processing-job, pipelines-overview, define-pipeline, model-registry, clarify-configure-processing-jobs, model-monitor, feature-store*, model-cards, model-dashboard, data-wrangler, autopilot-automate-model-development, mlflow; the SageMaker API reference (API_StoppingCondition, API_feature_store_PutRecord); the decision guides bedrock-or-sagemaker.html and genai-guide.html; the general maintenance services page; and the SageMaker pricing page.

---

Real-World Case Studies

AWS publishes these cases on aws.amazon.com/solutions/case-studies/ and aws.amazon.com/blogs/machine-learning/. They belong in this lesson because each one maps a lifecycle stage onto a service choice with a number attached; which is exactly how Task 1.3 questions are framed.

Case 1, HAYAT HOLDING: the whole train → deploy → monitor loop on a plant floor

Self-built ML environments were "time-consuming and cumbersome" at this MDF-panel manufacturer. The published pipeline: 194 sensors over OPC-UA → SiteWise Edge Gateway in AWS IoT Greengrass → SageMaker Model Training + Automatic Model Tuning + Model Deployment, with SageMaker Edge Manager running the model on-device. Result: $300,000 per year saved plus higher panel quality. Lifecycle lesson: stages 5 to 8 run fully managed, and Automatic Model Tuning replaces a manual hyperparameter sweep across 194 inputs. Source: aws.amazon.com/blogs/machine-learning/hayat-holding-uses-amazon-sagemaker-to-increase-product-quality-and-optimize-manufacturing-output-saving-300000-annually (2023).

Case 2, Forethought: DIY inference is a hidden tax

A 3-person team supporting 30 million interactions per year was running models and Kubernetes on its own Amazon EKS, memory exceptions and outages included. They migrated to SageMaker multi-model endpoints and Serverless Inference: −66 % cost on multi-model endpoints (with better latency), ≈ −80 % on serverless, and more than 80 % of GPU inference now runs on SageMaker. Lifecycle lesson: stage 7, shared endpoints and scale-to-zero beat a self-managed cluster you have to babysit. Source: aws.amazon.com/solutions/case-studies/forethought-technologies-case-study.

Case 3, RareJob: the bottleneck was orchestration, not model quality

Training on a local PC allowed one model per developer; the EC2/ECS attempt that followed still cost too much. AWS Glue and Amazon Athena feed parallel SageMaker Managed Spot Training jobs. Result: −25 % training time, > 10× developer efficiency, 100 hours per month saved, and speaking-test scores returned in 2 to 3 minutes. Lifecycle lesson: stages 1 to 3 (collect → EDA → pre-process) belong to Glue and Athena; the queue, not the algorithm, was the constraint. Source: aws.amazon.com/solutions/case-studies/rare-job-case-study (2020).

Case 4, Chronomics: when the in-house build fails, go AutoML

Four months of in-house custom computer-vision modelling never reached the target. Amazon Rekognition Custom Labels shipped in 3 to 4 weeks at 96.5 % accuracy / 97.9 % F1. Raising the decision threshold to 0.99 produced 99.6 % precision but discarded 5 % of predictions; 0.999 gave 99.87 % while discarding 27 %. Lifecycle lesson: stage 6: evaluation is what tells you to stop building, and the discarded tail needs a human review path. Source: aws.amazon.com/blogs/machine-learning/chronomics-detects-covid-19-test-results-with-amazon-rekognition-custom-labels (2022).

Case 5, Sun Finance: separate OCR from reasoning

The first attempt sent ID photos straight to Claude Sonnet 4 and scored 61.8 % overall (43 % on ID numbers): AWS rejected it. The rebuilt pipeline: Amazon Textract OCR → Amazon Rekognition fallback and face masking → Claude Sonnet 4 structuring → validation rules, with Amazon Titan Multimodal Embeddings in S3 Vectors for fraud similarity, evaluated on 585 images. Result: accuracy 79.73 % → 90.80 %, cost per document −91 %, processing time 20 hours → under 5 seconds, fraud detection 81 %. Lifecycle lesson: evaluation (stage 6) killed the elegant prototype, and the prebuilt service: not the model: owns extraction. Source: aws.amazon.com/blogs/machine-learning/sun-finance-automates-id-extraction-and-fraud-detection-with-generative-ai-on-aws (2026).

Case 6, Anthem: prebuilt first, humans in the tail

Manual claim-form extraction took 20 minutes per claim. An Amazon Textract pipeline (OCR plus ML index/classify behind the claims portal) automated 80 % of the workflow with a target of 90 %+, across thousands of claims per day. Lifecycle lesson: stages 1 to 2: call a managed API before training anything custom, and design the human exception path for the remaining 10 to 20 %. Source: aws.amazon.com/solutions/case-studies/anthem (2021).

All six cases in one table

Five rules that recur across all six cases:

1. Choose the tier deliberately: prebuilt service (Anthem, Chronomics) → managed custom ML (HAYAT, RareJob, Forethought) → generative AI (Sun Finance). The exam rewards the cheapest tier that works.
2. Baseline before build: a percentage without a "before" is marketing, not a metric.
3. Separate concerns: OCR is not reasoning (Sun Finance), and retrieval is not generation.
4. Evaluation is a phase, not a checkbox: Sun Finance's and Chronomics' first attempts both failed at stage 6.
5. Keep a human in the tail: the discarded 5 % and the un-automated 10 to 20 % still need someone.

📚 Did you know? The only production rate AWS publishes for its own Generative AI Innovation Center is 65 % of projects reaching production in 2025, some in as little as 45 days, out of more than 1,000 implementations, using AWS's Five V's framework: Value → Visualize → Validate → Verify → Venture. AWS has never claimed that all pilots succeed, so any option saying "100 %" is wrong on its face.

---

14. Practice Questions