Training and Hyperparameter Tuning on Amazon SageMaker

A training job is the most expensive object you create before a model ever serves a request, and it is the object the AIF-C01 exam asks you to reason about most precisely. Domain 1 carries 20% of the scored content, and Task 1.3 names "performing hyperparameter tuning or model optimization" outright. The failure mode is never "you did not know SageMaker can train". It is that you chose File mode for a 600 GB dataset of small files, or you picked Bayesian when the requirement was "reproduce the identical set of combinations on every re-run", or you enabled Spot without a checkpoint and then hit the undocumented-in-your-notes 3,600 second wait cap.

By the end of this lesson you will be able to:

• name the required fields of CreateTrainingJob and explain what each one controls;
• choose between File, FastFile and Pipe input modes using AWS's own size thresholds, and configure channels (s3_data_type, distribution, compression, record_wrapping);
• select an ml.* instance family for training versus inference, and compute cluster size as instances × GPUs;
• decide between data parallelism and model parallelism, and name the AWS libraries (SMDDP, SMP v2) that implement them;
• run Automatic Model Tuning and pick the right strategy: Grid vs. Random vs. Bayesian vs. Hyperband;
• configure Managed Spot Training correctly (MaxWaitTimeInSeconds ≥ MaxRuntimeInSeconds) and compute the actual savings with the official formula;
• size a checkpoint workflow so an interrupted job resumes instead of restarting;
• read CloudWatch training metrics, and explain what Debugger and Experiments do, and that Debugger is closed to new customers;
• defend a comparative verdict on Grid vs. Random vs. Bayesian and on managed training vs. do-it-yourself;
• answer ten exam-style questions written in the official voice, including two cost calculations.

---

1. Anatomy of a SageMaker training job

1.1 The eight required pieces

CreateTrainingJob is a single API call, but it is a bundle of eight independent decisions. The exam tends to test them as "which of these is missing / misconfigured", so learn them as a checklist.

Two limits are worth memorizing because they are cheap to recall and appear often: a single training job's MaxRuntimeInSeconds defaults to 1 day and hard-stops at 28 days, and across running plus waiting the job cannot exceed 30 days. On stop, SageMaker sends SIGTERM and waits 120 seconds for your code to flush; which is exactly the window your checkpoint handler has to work in.

1.2 The lifecycle a job actually goes through

1.3 What training costs, and how it is billed

SageMaker bills per instance-second for partial instance-hours, with no minimum and no upfront commitment. That matters more than any single hourly rate: a 90-second training job costs 90 seconds of an instance, not a full hour.

• 📚 Did you know? The SageMaker free tier for training is 50 hours of ml.m4.xlarge or ml.m5.xlarge in the first two months: enough for roughly six full working days of continuous CPU training, but only a handful of GPU runs. AWS labels the two-month window as a promotion on the pricing page, so confirm the terms before you budget a course around it.

---

2. Getting data into the job: input modes and channels

This is the highest-yield topic in the lesson. AWS documents exactly three input modes in the API enum, File, FastFile and Pipe, and the developer guide still calls File the default.

2.1 The three modes side by side

2.2 File mode: the default that silently costs you time

In File mode the training instance must have enough storage space to fit the entire dataset; that is a direct quote from the developer guide, and it is the mechanism behind the classic exam item: "training starts only after a long data-transfer phase and the instance storage must hold the whole dataset." Answer: File.

File mode is also the only mode the docs explicitly tie to incremental training, and it accepts all three s3_data_type values, including AugmentedManifestFile. Sharding across instances is done with distribution = ShardedByS3Key.

AWS's own sizing guidance: File mode suits datasets below 50 to 100 GB, and at the documented rate of 50 GB in about 5 minutes with 100 MB shards, AWS advises serializing files smaller than 50 MB into chunks of roughly 150 MB (TFRecord or RecordIO).

Worked example 1, File-mode sizing (section 12, Example 1). An 80 GB dataset in File mode needs a volume ≥ 80 GB plus room for output. At AWS's rate the download takes about 8 minutes before step one runs, roughly 21% of a 30-minute job spent copying. Switch to FastFile, or shard with ShardedByS3Key.

2.3 FastFile: stream without copying

FastFile identifies the objects but downloads nothing. S3 is exposed as a POSIX filesystem and read on demand, so startup is fast and local space is only needed for artifacts. Two constraints to memorize: it accepts prefix only (no augmented manifest), and random reads work but sequential reads are best. Extra CloudTrail and KMS charges can apply because FastFile records object-level access.

2.4 Pipe: the original streaming mode

Pipe streams S3 objects into a FIFO pipe with high pre-fetch, keeps disk only for artifacts, and gives you managed per-epoch shuffling, Gzip compression and RecordIO wrapping. AWS now describes it as "largely replaced by … fast file mode", so on the exam Pipe is usually the streaming distractor when FastFile is the better answer, or the correct answer when the question mentions RecordIO/Gzip/per-epoch shuffle.

2.5 Channels: how objects are enumerated and split

A channel is one input to the job. Its configuration fields are where the exam hides detail.

Supported stores are Amazon S3, Amazon EFS and Amazon FSx for Lustre (plus S3 Express One Zone), and the dataset must live in the same Region as the training job: an S3 bucket in us-west-2 feeding a job in us-east-1 is not a supported pattern.

2.6 Choosing a mode, using AWS's thresholds

---

3. Choosing compute: the ml.* instance families

3.1 The naming rule

Every SageMaker compute SKU is prefixed with ml.. That prefix is the exam's cheapest tell: if an option says p4d.24xlarge without the ml. when the context is a SageMaker training job or endpoint, it is describing EC2, not SageMaker. The families split cleanly by role.

3.2 The GPU hardware AWS publishes

3.3 Two sizing rules that decide your bill

Rule 1: scale the instance size before you add instances. AWS's distributed-training guidance is explicit: a single ml.p3.16xlarge beats two ml.p3.8xlarge for the same 8 GPUs, because you pay for one interconnect instead of a network hop. Adding instances is what you do when one machine cannot hold the model or the data.

Rule 2: cluster size = instances × GPUs. Two ml.p3.16xlarge nodes give you a 16-GPU cluster, not 8. Every parallelism degree, batch-size calculation and communication cost in section 4 is derived from that product.

Worked example 2: distributed sizing (section 12, Example 2). ml.p3.16xlarge × 2 = 8 GPUs per node → cluster size 16. AWS's published reference for a 70B-parameter LLM is 32 × ml.p4d.24xlarge = 256 GPUs, with sharded-data-parallel degree 256 and batch size 2; a 175B model needs 64 × ml.p4d.24xlarge, while a 7B model fits on 1 × ml.p4d.24xlarge.

• 📚 Did you know? ml.p5.48xlarge ships 8 × H100 with 640 GB of total GPU memory and 3,200 Gbps of EFAv2 networking: more network bandwidth than most racks had a decade ago, and the reason multi-node training on that family is bandwidth-bound rather than compute-bound. It reached general availability on 4 August 2023, initially in us-east-1 and us-west-2.

---

4. Distributed training: data parallelism vs. model parallelism

4.1 The one distinction the exam wants

The exam phrasing is usually a symptom: "the model does not fit in a single GPU's memory, but the dataset is modest" → model parallelism. The mirror image: "the model trains fine on one GPU but the dataset is enormous" → data parallelism.

4.2 The libraries AWS names

For large multi-node jobs, AWS's guidance converges on the same stack every time: EFA-enabled GPUs, ideally ml.p4d/ml.p4de (A100) or newer, plus Amazon FSx for Lustre for the dataset. EFA matters because it bypasses the OS network stack; FSx matters because a thousand small files read over S3 will starve 256 GPUs.

Worked example 3: how much model fits where (section 12, Example 3). AWS's published references: Llama 2 7B → 1 × ml.p4d.24xlarge; Llama 2 70B → 32 × ml.p4d.24xlarge; 175B → 64 ×; GPT-2 10B → 16 ×. Read them as a rule of thumb: parameter count, not dataset size, drives the instance count once you cross single-GPU capacity.

---

5. Automatic Model Tuning: what a tuning job does

5.1 The idea in one sentence

Automatic Model Tuning (AMT) runs many training jobs over the hyperparameter ranges you supply and keeps the one that performs best on your objective metric. You define ranges; SageMaker runs the search; you get the best trial back.

The exam calls this "hyperparameter tuning or model optimization" and it is squarely inside Task 1.3.

5.2 The tuning-job flow

5.3 Ranges, scaling and the objective metric

Hyperparameter ranges are typed and scaled. Auto, Linear, Logarithmic and ReverseLogarithmic are the four scaling options, and each has a domain rule worth knowing because it is a free correctness check on the exam:

TrainingJobEarlyStoppingType is OFF by default and can be set to AUTO to stop trials that are clearly not going to win. Warm start reuses the parent trials of a previous tuning job, which cuts both cost and wall-clock time when you extend a search rather than restart it.

5.4 The limits you should be able to recite

Two consequences follow immediately. First, AWS advises keeping the hyperparameter count well under the 30 cap: each extra dimension dilutes the search. Second, RandomSeed makes a Random or Hyperband search up to 100% reproducible, which is the mechanism behind the "reproduce identical combinations" exam item.

Worked example 4: tuning budget and quota arithmetic (section 12, Example 4). MaxNumberOfTrainingJobs = 20, one hour each on ml.g4dn.12xlarge at AWS's published $4.89/h → 20 × $4.89 = $97.80. With MaxParallelTrainingJobs = 10 you run 2 waves ≈ 2 hours instead of 20. Pushing to 100 jobs costs $489 with no guarantee of better accuracy. Separately, AWS's own quota example: 10 tuning jobs × 100 training jobs × 20 concurrent on ml.m4.xlarge passes the 100-concurrent-tuning and 750-job limits, but needs a raise for 20 concurrent training jobs (default 10) and 200 ml.m4.xlarge instances (default 20).

• 📚 Did you know? A SageMaker tuning job is not a separate compute product; it is an orchestrator that creates ordinary training jobs and watches their CloudWatch metrics. That is why tuning-job quotas are expressed both in tuning jobs (100 concurrent) and in training jobs (10 parallel, 750 total): you are metered on the same instance-seconds either way.

---

6. Tuning strategies: Grid vs. Random vs. Bayesian vs. Hyperband

This is the comparison the exam asks you to make under time pressure. Learn the row that differs, not the whole table.

6.1 The full comparison

6.2 Grid: deterministic, categorical, combinatorial

Grid enumerates every distinct combination of the categorical values you give it, and SageMaker computes the job count automatically; you should set MaxNumberOfTrainingJobs to that number. It is the strategy for reproducing results and for even coverage of a search space.

Worked example 5: Grid job count (section 12, Example 5). Two categorical parameters with 5 and 4 values → 5 × 4 = 20 combinations → SageMaker creates 20 training jobs. Add a third parameter with 3 values → 60 jobs. Grid explodes combinatorially, which is exactly why AWS steers large search spaces to Bayesian or Random.

6.3 Random: maximum parallelism, zero learning

Because Random ignores prior results, every trial is independent and the strategy supports the largest number of parallel training jobs of any strategy. It is also the only strategy whose job cap can be raised from 750 to 10,000, and with a fixed RandomSeed it reproduces the identical sequence of configurations.

6.4 Bayesian: the SDK default, and a sequential optimizer

Bayesian builds a regression model over the trials completed so far and proposes the next configuration where improvement looks most likely. That makes it the best use of a small budget and the default in the SageMaker SDK's HyperparameterTuner, but it also makes it sequential: trial n+1 depends on trial n, so AWS states it cannot massively scale in parallel.

6.5 Hyperband: multi-fidelity, iterative algorithms only

Hyperband reallocates epochs between successive rounds, stops weak jobs early and runs many jobs in parallel. The hard constraint is the algorithm: it needs iterative algorithms that can be meaningfully judged part-way through. XGBoost and Random Cut Forest qualify; decision trees and k-NN do not. AWS markets it as up to 3× faster than Bayesian; that figure comes from an AWS what-is page, so cite it as an AWS claim rather than a reproduced benchmark.

---

7. Managed Spot Training and checkpoints

7.1 What it is

Managed Spot Training runs your training job on EC2 Spot capacity with SageMaker managing the interruptions for you. AWS launched it on 26 August 2019 and has published two different savings figures for it ever since: both are AWS-published, and both appear in exam material:

7.2 The official formula, and the only calculation you need

$$\text{Savings (\%)} = \left(1 - \frac{\text{BillableTimeInSeconds}}{\text{TrainingTimeInSeconds}}\right) \times 100$$

Worked example 6: Spot savings math (section 12, Example 6). A job runs for 500 seconds and is billed for 100 seconds:

$$(1 - 100/500) \times 100 = 80\%$$

The AWS launch blog's demonstration job ran 2,423 seconds and was billed for 837 → $(1 - 837/2423) \times 100 \approx \mathbf{65\%}$. Compute it per job; never assume one number.

Worked example 7, On-Demand vs. Spot for one GPU run (section 12, Example 7). AWS-published ml.g4dn.12xlarge = $4.89/h. Two hours On-Demand = $9.78. If only 36 of 120 wall-clock minutes are billable on Spot: $0.6 \times 4.89 \approx$ $2.93, about 70% saved, matching the Cinnamon AI customer story.

7.3 The two configuration rules

Worked example 8, Spot configuration (section 12, Example 8). A 6-hour checkpointing job: MaxRuntimeInSeconds = 21,600 and MaxWaitTimeInSeconds larger than that (AWS's demo used 48 h = 172,800 s), plus CheckpointConfig.S3Uri. Run the same 6-hour job without checkpointing and the wait cap is 3,600 s, so you either add checkpointing or run On-Demand.

7.4 Checkpoints: resume, not restart

7.5 What an interrupted job actually looks like

The documented state machine is Starting → Downloading → Training → Uploading, with Interrupted → Starting → … loops in the middle and a terminal Stopped: MaxWaitTimeExceeded when the wait budget runs out before the job finishes.

---

8. Observing the run: metrics, Debugger and Experiments

8.1 Where the numbers go

Training metrics are published to Amazon CloudWatch under the namespace /aws/sagemaker/TrainingJobs, and the SageMaker console's Training jobs → Monitor view graphs them. This is the answer whenever a question asks how you know whether a job is converging, diverging or thrashing the GPU.

8.2 Debugger: tensors, rules and actions

So the exam item "automatically stop the training job when a vanishing-gradient rule fires and email the team" resolves to Debugger built-in actions: with the caveat that Debugger is closed to new customers.

8.3 SageMaker Experiments

Experiments tracks every trial and run, the hyperparameters used, the metrics produced, the artifacts written, so you can compare runs side by side. Debugger output can be routed to TensorBoard for visualization. Experiments is the "which of my 20 runs was best, and with what settings" answer; Debugger is the "what is happening inside this run right now" answer.

---

9. Paths that skip the training loop entirely

9.1 JumpStart: pre-trained models you can fine-tune

JumpStart is SageMaker's hub of pre-trained open-source models drawn from TensorFlow Hub, PyTorch Hub, Hugging Face and GluonCV. Two numbers matter for the exam: it covers 15 problem types, of which 8 are trainable, meaning you can do incremental training / fine-tuning on them, and it integrates with Automatic Model Tuning.

9.2 Autopilot: AutoML for tabular (and, via v2, more)

Scope: tabular data up to hundreds of gigabytes; text, image and forecasting tasks plus LLM fine-tuning are handled through the AutoML v2 API. The UI moved into SageMaker Canvas on 30 November 2023, so no-code users go to Canvas and Autopilot remains the API path.

9.3 Choosing a starting point

• 📚 Did you know? Autopilot's algorithm shortlist is deliberately small, gradient-boosted trees, a feedforward DNN and logistic regression, because AutoML is a selection and tuning engine, not a research lab. The HPO stage it runs internally is the same Automatic Model Tuning machinery described in section 5, which is why the two share quota language.

---

10. Comparative verdict: strategies and managed vs. DIY

10.1 Managed training vs. building it yourself

10.2 Tuning strategy at a glance

---

11. Limits, quotas and cost evidence in one place

11.1 Published limits and benchmarks

11.2 Cost evidence

11.3 2025 to 2026 Updates

Every number in sections 1 to 10 comes from AWS pages checked on 6 October 2026. What actually moved in 2025 to 2026 is mostly availability, not arithmetic: the product was renamed, a whole band of SageMaker features around the training loop was placed into maintenance, and the exam guide was revised. Availability changes matter on the exam because a feature that is closed to new customers can be the wrong answer even when the developer guide still documents it in full.

AWS's own lifecycle vocabulary (AWS General Reference, Service lifecycle) is worth learning because it tells you what "closed" actually means: Maintenance = no new customers, no new features, still supported; Sunset = a planned end of operations, typically about 12 months out; Full Shutdown = removed from the portfolio. Every SageMaker item on the 30 July 2026 notice sits in the first state, not the third; which is exactly why this lesson keeps Debugger in section 8 with a date attached instead of deleting it.

• 📚 Did you know? AWS states that exam-guide changes appear on the live exam about one month after publication, so the v1.1 objectives published 30 April 2026 became examinable from roughly late May 2026. That same revision put SageMaker JumpStart on the in-scope service list (section 9.1 of this lesson), dropped Amazon MemoryDB, and left the exam's arithmetic untouched: 65 questions (50 scored + 15 unscored), 90 minutes, USD 100, pass 700/1000, three-year validity, with domains weighted 20 / 24 / 28 / 14 / 14.

---

12. Worked AWS examples with numbers

Example 1: File-mode sizing. 80 GB dataset, File mode → volume ≥ 80 GB plus output headroom. At AWS's documented rate (50 GB ≈ 5 min at 100 MB shards) the copy takes ≈ 8 minutes, about 21% of a 30-minute job, before the first training step. Fix: FastFile, or distribution = ShardedByS3Key.

Example 2: Distributed sizing. ml.p3.16xlarge × 2 → 8 GPUs/node → cluster size 16. AWS guidance: try the larger single node first (1 × 16xlarge beats 2 × 8xlarge for the same 8 GPUs). Reference scale: 7B → 1 × ml.p4d.24xlarge, 70B → 32 × (256 GPUs, sharded-data-parallel degree 256, batch 2), 175B → 64 ×.

Example 3, Model fit drives instance count. Parameter count, not dataset size, decides the node count once you cross single-GPU memory: 8 × A100 40 GB = 320 GB on one ml.p4d.24xlarge, 8 × H100 = 640 GB on one ml.p5.48xlarge. If the model plus optimizer state exceeds the device, no amount of data parallelism helps, you need model parallelism (SMP v2).

Example 4: Tuning budget and quota arithmetic. MaxNumberOfTrainingJobs = 20 × 1 h × ml.g4dn.12xlarge @ $4.89/h = $97.80; with MaxParallelTrainingJobs = 10 → 2 waves ≈ 2 h instead of 20 h. At 100 jobs the bill is $489 with no accuracy guarantee. AWS's quota example: 10 tuning jobs × 100 training jobs × 20 concurrent → passes the 100 tuning-job and 750 training-job limits, but needs raises for 20 concurrent training jobs (default 10) and 200 ml.m4.xlarge instances (default 20).

Example 5: Grid job count. 5 values × 4 values = 20 jobs. Add a third parameter with 3 values → 60 jobs. Grid grows multiplicatively; that is the argument for Bayesian or Random on any non-trivial space.

Example 6, Spot savings. 500 s training, 100 s billed → $(1 - 100/500) \times 100 = \mathbf{80\%}$. Launch blog job: 837 / 2,423 → ≈ 65%. The same job at a 90% billable fraction would save only 10%, savings is a per-job measurement, never a fixed discount.

Example 7: On-Demand vs. Spot. ml.g4dn.12xlarge at AWS's published $4.89/h: 2 h On-Demand = $9.78. Spot with 36 billable minutes of 120 → $0.6 \times 4.89 \approx$ $2.93, i.e. ≈ 70% saved.

Example 8: Spot + checkpoint configuration. 6-hour job: MaxRuntimeInSeconds = 21,600, MaxWaitTimeInSeconds greater (AWS's demo used 48 h), plus CheckpointConfig.S3Uri and code writing to /opt/ml/checkpoints. Same job with a built-in algorithm and no checkpointing → wait capped at 3,600 s → run On-Demand instead.

• 📚 Did you know? SageMaker bills training by the second with no minimum, so the cheapest failed experiment is a fast one. That is the real argument behind TrainingJobEarlyStoppingType = AUTO: stopping a hopeless trial at minute 12 instead of minute 60 is a 5/6 reduction in that trial's bill, and with up to 100 parallel training jobs per tuning job those savings multiply across the whole search.

---

13. Conflicting, unverified and non-examinable facts

Separate what AWS documents from what AWS markets. Keep this table out of your memorization set except where it is marked "use for the exam".

Primary sources for this lesson: the SageMaker developer guide pages model-access-training-data, model-access-training-data-best-practices, automatic-model-tuning{,-how-it-works,-considerations,-limits}, multiple-algorithm-hpo-create-tuning-jobs, model-managed-spot-training{,-status}, model-checkpoints{,-resume}, distributed-training{,-get-started,-scenarios}, model-parallel-intro-v2, model-parallel-best-practices-v2, train-debugger, debugger-debug-training-jobs, debugger-built-in-actions, experiments-mlops, view-train-metrics, cmn-info-instance-types, jumpstart-models, studio-jumpstart, use-auto-ml, autopilot-automate-model-development; the API reference API_CreateTrainingJob, API_AlgorithmSpecification, API_Channel, API_HyperParameterTuningJobConfig, API_ResourceLimits, API_StoppingCondition; boto3 create_training_job / describe_training_job; the SageMaker SDK HyperparameterTuner; the AIF-C01 Exam Guide; and the SageMaker pricing pages.

---

14. Real-World Case Studies

AWS publishes few customer stories about training as such, the billable moment is rarely photogenic, so these four are the training-and-tuning half of the AWS case-study record: each one names its services, its numbers and its source, and each maps back onto a section above. Read every percentage as a customer- or AWS-claimed, unaudited figure, and read "up to" as a ceiling.

14.1 The four stories

HAYAT HOLDING: Automatic Model Tuning on a live production line (manufacturing, 2023). The MDF-panel maker had built its own ML environments and described them as "time-consuming and cumbersome". The published architecture runs 194 sensors over OPC-UA into an AWS IoT Greengrass SiteWise Edge Gateway, then SageMaker Model Training + Automatic Model Tuning + Model Deployment, with SageMaker Edge Manager serving the resulting model on the device. Reported outcome: $300,000 per year saved, alongside higher quality and optimised output. This is section 5 in production: manual hyperparameter sweeps replaced by AMT across 194 inputs: with the deployment target an edge device rather than an endpoint. Source: AWS Machine Learning Blog, blogs/machine-learning/hayat-holding-uses-amazon-sagemaker-to-increase-product-quality-and-optimize-manufacturing-output-saving-300000-annually (2023).

RareJob: managed Spot training instead of a queue (EdTech, 2020). Speaking-test scoring (PROGOS) started on a local PC, which allowed one model per developer, then moved to EC2/ECS and still cost too much. The AWS architecture feeds AWS Glue and Amazon Athena into parallel SageMaker managed Spot training jobs. Reported outcome: 25% less training time, more than 10× development efficiency, 100 hours per month saved, and scores returned in 2 to 3 minutes. It is section 7 in production: the bottleneck was queueing, not model quality, and Spot plus orchestration turned a serial queue into a fan-out. Source: AWS case study, solutions/case-studies/rare-job-case-study (2020).

Chronomics, the documented DIY-training failure (health-tech, 2022). Four months of in-house custom computer-vision modelling for reading COVID-19 test results never reached target. The team shipped with Amazon Rekognition Custom Labels (AWS AutoML) in 3 to 4 weeks at 96.5% accuracy / 97.9% F1, scored with DetectCustomLabels. The published threshold analysis is the part worth memorising: a confidence threshold of 0.99 → 99.6% of predictions correct with 5% discarded, and 0.999 → 99.87% with 27% discarded. The lesson is tier selection (section 9): for a narrow vision task, managed AutoML beat a hand-built training loop, and somebody still has to decide what happens to the discarded 5%, the human path AWS documents for that tail is Amazon A2I. Source: AWS Machine Learning Blog, blogs/machine-learning/chronomics-detects-covid-19-test-results-with-amazon-rekognition-custom-labels (2022).

Forethought, what DIY infrastructure costs a three-person team (SaaS customer support). SupportGPT handles 30 million interactions per year with several models per customer, and the 3-person team could not run both models and Kubernetes, memory exceptions and outages drove them off their own Amazon EKS. They migrated to SageMaker multi-model endpoints and SageMaker Serverless Inference: 66% lower cost on multi-model endpoints with better latency, approximately 80% lower on Serverless (headline: up to 80%), and more than 80% of GPU inference now running on SageMaker. The tie-in to this lesson is the section 10 verdict: every dollar saved on serving is a dollar back into the training and tuning budget, and self-managed infrastructure is the hidden tax both columns of that table warn about. Source: AWS case study, solutions/case-studies/forethought-technologies-case-study.

14.2 Services, numbers and sources at a glance

• 📚 Did you know? The sharpest "before" in the whole case-study record belongs to Sun Finance (fintech, ML Blog 2026): its first prototype sent ID photos straight to Claude Sonnet 4 and scored 61.8% overall, 43% on the ID number, and was rejected. Separating the jobs (Amazon Textract for OCR, Rekognition as fallback and face masking, the model only for structuring, plus validation rules on 585 images) took the same pipeline to 85% and then 90.80% accuracy, −91% cost per document and 20 hours → under 5 seconds. The training loop was never the problem; the evaluation was.

14.3 What each case proves against this lesson

---

Practice Questions

---