Choosing Data Stores and Storage Formats

Ask ten candidates how they choose a database and you will hear ten versions of "it depends", and on DEA-C01 that answer scores nothing. The exam does not reward vague architecture taste; it rewards a deterministic mapping: one access pattern in the stem, one service out of the in-scope list, one format at rest, one compression codec. Domain 2 is worth 26% of the exam, and task 2.1 is the store-selection core of that domain, skills 2.1.1 (implement storage for cost and performance), 2.1.2 (configure storage for access patterns), 2.1.3 (apply stores to use cases), 2.1.7 (manage open table formats, for example Apache Iceberg) and 2.1.8 (describe vector index types, for example HNSW and IVF).

In this lesson you will:

• read task 2.1 skill by skill and build the in-scope store taxonomy the exam draws from;
• map access pattern → service with a master table and a decision tree you can walk in seconds;
• size DynamoDB with real capacity-unit arithmetic and its 400 KB / 10 GB ceilings;
• place RDS, Aurora, Neptune, MemoryDB and DocumentDB so you never pick two services for one pattern;
• separate Redshift, Athena and OpenSearch by what the query actually does;
• pick a format from the CSV / JSON / Avro / Parquet / ORC matrix: layout, schema, evolution and when;
• choose a compression codec from AWS's own wording (Snappy, GZIP, ZSTD, ZLIB) and the defaults that become distractors;
• run S3 as the lake core: 3,500 / 5,500 requests per partitioned prefix, no prefix limit, strong consistency since 1 December 2020, and the hot-prefix 503 fix;
• reason about lakehouse table formats, Iceberg, Hudi and Delta, to exactly the depth skill 2.1.7 requires;
• apply decision tables to exam scenarios, and split Athena versus Redshift at the consumption layer;
• study two AWS customer case studies and a sourced October 2026 update box;
• practise with 12 exam-style questions plus three interactive checks.

---

1. What task 2.1 actually tests

1.1 The five skills behind this lesson

Two boundaries matter more than the lists themselves. Engine deep-tuning is out: Redshift internals belong to Lesson 4, and orchestration to Lesson 5. What is left here is selection: pattern in, service out, format attached.

1.2 The in-scope store taxonomy

Everything you may name as the correct answer comes from this table (DEA-C01 in-scope lists, checked Oct 2026):

• 📚 Did you know? Amazon ElastiCache and Amazon Timestream sit on neither the in-scope nor the out-of-scope list (both lists checked Oct 2026), while skill 2.1.3 names MemoryDB by name. The practical reading: when a stem uses the guide's own "fast key/value access" wording, MemoryDB is the in-scope pick, and ElastiCache remains the real-world cache answer you will meet in production diagrams. Never spend exam minutes arguing about list hygiene; answer with the service the stem's wording matches.

---

2. Access pattern → service: the master map

2.1 The table you should be able to recite

Teaching rule:

• *key + latency → DynamoDB or MemoryDB
• joins and BI → Redshift
• SQL on the lake → Athena
• text → OpenSearch
• relationships → Neptune
• OLTP → RDS or Aurora
• cheap raw scale → S3.*

2.2 Worked example E1: one system, two stores

A retail service has two requirements in the same ticket:

The stem always contains two requirements on purpose. Split the workload; never force one store to answer a pattern it was not designed for.

2.3 Interactive check: match pattern to service

---

3. Key-value, document, graph and in-memory stores

3.1 DynamoDB: the constraints the exam quotes

DynamoDB's exam identity is single-digit-millisecond key access at any scale, and its deliberate silence about JOINs is the point. Any option proposing DynamoDB for lake-wide aggregation has misread the service.

3.2 Worked example E2: capacity units by hand

3.3 Worked example E3: the 400 KB item headroom test

The exam's favourite version of this question puts the number just over 400 KB in the stem. The answer is never "increase the item limit"; there is no such knob.

3.4 The neighbours: DocumentDB, MemoryDB, Neptune

• 📚 Did you know? MemoryDB's claim to exam fame is a single sentence in skill 2.1.3, "MemoryDB for fast key/value access", which is why it beats every other cache-flavoured option when the stem asks for Redis structures plus durability. A plain cache in front of a database is a real architecture, but the guide's named use case is the durable one: if the data must survive a node failure without a rebuild from the source of truth, you are reading the MemoryDB line.

---

4. Relational OLTP: Amazon RDS and Amazon Aurora

4.1 RDS is the documented default

Amazon RDS presents the exam's default managed relational choice: you choose an engine, Multi-AZ for failover, read replicas for scale-out, and automated backups: AWS takes the OS and engine patching. The service's own welcome page recommends RDS as your default choice for relational workloads, which is why any stem describing conventional OLTP (transactions, joins, foreign keys, a schema that is enforced on write) resolves to RDS unless a specific Aurora capability is named.

4.2 Aurora: the deltas the stem names

The vector line is the newest trap in this domain: HNSW (hierarchical navigable small world, recall-oriented, memory-hungry) and IVF (inverted file, faster build, cluster-based search) are index types, not services. The stem names the index, the option names Aurora PostgreSQL.

4.3 Worked example E4; which relational flavour?

Every stem hides a shape word: transaction, foreign key, failover = OLTP; extract, join across many tables, analysts = OLAP; vector, embedding, similarity = Aurora PostgreSQL with HNSW/IVF; keyed by, by ID, single-digit ms = DynamoDB.

---

5. Analytics stores: Redshift, Athena and OpenSearch

5.1 Three analytics shapes, three services

AWS's own framing is quotable: Athena is for "interactive ad hoc SQL queries against data on Amazon S3, without … infrastructure or clusters", while Redshift is "optimized … on complex queries that join large numbers of very large database tables".

5.2 OpenSearch: the log-analytics identity

The split that appears in stems: aggregating 5xx errors by service and full-text drilling into a stack trace is one query shape, OpenSearch. "Join the error counts to the customer dimension table" is a different shape, Athena or Redshift.

---

6. The storage-format matrix

6.1 CSV, JSON, Avro, Parquet, ORC, and Protobuf for completeness

Three comparisons the exam reuses endlessly:

6.2 Predicate pushdown in one picture

Athena's documentation states it plainly: "Predicate pushdown in Parquet and ORC enables Athena queries to fetch only the blocks it needs… it uses statistics from data block predicates, such as max/min values, to determine whether to read or skip the block." A file of daily totals where the query asks for one date lets the engine skip whole row groups; that skipping is the cost model, not a micro-optimisation.

6.3 Sizing defaults worth memorising

6.4 Worked example E5: AWS's own cost ladder

Athena's pricing page carries this exact worked example (as of Oct 2026):

Lesson for the exam: compression alone gives the first cut; columnar layout gives the second; neither is optional, and neither changes the query. When a stem says "reduce query cost without changing the BI tool", the answer is convert to columnar: not buy a bigger cluster.

---

7. Compression codecs: Snappy, GZIP, ZSTD, ZLIB

7.1 AWS's own wording

The governing principle from Athena's tuning guidance: higher ratio costs CPU. Compress the lake, then let columnar layout plus predicate pushdown cut the I/O.

7.2 Defaults that become three-way distractors

Learn them as a triple: any question that pairs "which compression does X use by default" with three of these four names is testing recall of exactly this mapping.

7.3 Worked example E6: the codec decision in a stem

7.4 Worked example E7: conversion arithmetic

• 📚 Did you know? Amazon Athena infers compression from the file extension for text and JSON formats, so renaming a .csv.gz to .csv does not "decompress" anything, it just tells the engine to expect plain text it will not find. That is why format questions and codec questions are separate exam items: the format is what you declare, the codec is largely what the engine discovers.

---

8. Amazon S3 as the data-lake core

8.1 Request rates: per prefix, not per bucket

AWS's performance documentation (accessed Oct 2026) states:

"Your application can achieve at least 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per partitioned Amazon S3 prefix. There are no limits to the number of prefixes in a bucket."

8.2 Worked example E8: the hot-prefix fix

8.3 Partitions and prefixes are different axes

Over-partitioning produces small files and planner overhead ("reduce your request rate" style errors appear in Athena tuning guidance at tiny split sizes). The rule: partition for the WHERE clause, spread prefixes for the write rate, and target roughly 128 MB splits.

8.4 Consistency: the 2020 change that killed two answers

AWS's announcement of 1 December 2020:

"Effective immediately, all S3 GET, PUT, and LIST operations, as well as operations that change object tags, ACLs, or metadata, are now strongly consistent."

• Free, applies to all Regions and all objects.

• Consequences: EMRFS Consistent View and S3Guard are obsolete: any option whose correctness requires them is stale and wrong.

• A "read-after-write is eventually consistent in S3" option has been false for every exam since 2021.

• 📚 Did you know? S3's storage-class ladder is itself a cost lever for the lake: Intelligent-Tiering moves objects untouched for 30 days to Standard-IA (−40%), 90 days to Glacier Instant Retrieval (−68%), and optionally 180 days to Deep Archive (−95%), while S3 Express One Zone trades a single Availability Zone for single-digit-millisecond latency at up to 2,000,000 requests/s per directory bucket (as of Oct 2026). Choosing the class is task 2.1's cost/performance half; choosing the format is this lesson's Section 6.

• 📚 Did you know? Amazon S3 has been a public service since 13 March 2006, and AWS's own 20-year retrospective (2026-03-13) reports a price reduction of approximately 85% since launch plus more than $6 billion saved collectively by customers who chose S3 Intelligent-Tiering instead of S3 Standard: together with the reminder that "the code you wrote for S3 in 2006 still works today, unchanged." That is why Intelligent-Tiering is the lake default when the re-read pattern is unknown: AWS moves an object untouched for 30 days to Standard-IA (−40%), 90 days to Glacier Instant Retrieval (−68%) and optionally 180 days to Deep Archive (−95%), so the cost decision runs itself while the format decision (Section 6) stays yours.

---

9. The lakehouse layer: Iceberg, Hudi and Delta

9.1 Why table formats exist

File formats "only allow for writes of entire files", so change-data-capture, deletes and continuous streaming create small-file pain and offer no transactional semantics on object storage. Table formats add snapshots, compaction, time travel and transactional semantics on top of Parquet/ORC/Avro files in S3. A plain data lake, AWS's Prescriptive Guidance notes, doesn't provide support for atomicity, consistency, isolation, and durability (ACID) until a table format such as Iceberg is added.

Iceberg is the one the guide names (skill 2.1.7), so it is the one you must know in depth:

9.2 Hudi and Delta: awareness, not internals

AWS's own guidance says "no single table format is a 'one size fits all'". The comparison matrix AWS publishes (accessed Oct 2026):

Because the guide names only Iceberg, do not memorise internals of the other two: recognise the names and know that AWS declines to crown a winner.

9.3 Schema-on-read versus schema-on-write

Exam phrasing: lake = schema-on-read, cheap storage, decoupled compute; warehouse = schema-on-write, curated, enforced, tuned for BI. Fixed dashboards with heavy joins → write; raw, variable structures → read.

9.4 Worked example E9: hidden partitioning and time travel

---

10. Decision tables for exam scenarios

10.1 The master scenario table

10.2 Format lifecycle: interactive ordering

10.3 The ten trap pairs

---

11. Cross-service consumption: Athena versus Redshift

Store selection does not end at storage: the exam asks which engine reads it. AWS's comparison guidance (accessed Oct 2026) is unambiguous: Athena = interactive ad-hoc ANSI SQL on S3 with no clusters; Redshift = the warehouse; EMR = full Spark/Hive.

Cost levers that show up in stems, in order of impact:

1. Columnar format + codec: AWS's own $15 →$5 → $1.25 ladder.
2. Partition pruning: only helps when the WHERE uses the partition key; SELECT * throws column savings away.
3. Partition projection: Athena derives partitions from table properties instead of calling GetPartitions; AWS reports query speedups such as 137 s → 10 s (~92%).
4. Workgroup bytes-scanned cutoff: BytesScannedCutoffPerQuery has a 10,000,000-byte minimum, capping worst-case spend per query.
5. Query result reuse: default 60 minutes, maximum 7 days, per workgroup.

11.1 Worked example E10: pruning math

2026 Updates (as of October 2026)

• 📚 Did you know? The AWS Glue Schema Registry: the component that makes Avro-in-motion safe: is serverless and free, and validates exactly three formats: AVRO (1.11.4), JSON and PROTOBUF. Its Avro model is the one sentence worth memorising: the schema is JSON, the data is binary; which is why an Avro file is readable only with its schema, and why "the schema travels with the data" is the correct exam explanation for Avro's excellent evolution behaviour.

---

Real-World Case Studies

AWS publishes what these abstractions look like in production. Every figure below is customer- or AWS-claimed and unaudited, with the source named so you can check it: the examinable point is the pattern (which store, which format, which number moved), not the marketing.

Case A, Nasdaq: an S3 write path and a Redshift read path

"We were able to easily support the jump from 30 billion records to 70 billion records a day because of the flexibility and scalability of Amazon S3 and Amazon Redshift.": Robert Hunt, VP Software Engineering, Nasdaq (AWS case study, accessed Oct 2026)

Read Nasdaq as one system, two stores: S3 absorbs the write spike cheaply and durably, Redshift serves the concurrent reads, and Spectrum closes the gap so the 15 TB lake is never copied. The phrase AWS's page highlights is "zero contention between data loading and querying"; that is the decoupled-storage argument of Section 9, visible in a production deadline.

Case B, EMX: S3 plus Athena instead of a backend warehouse

EMX is the working version of Section 6: keep raw data in S3, convert to a columnar layout, and let Athena's per-terabyte metering replace an always-on backend. The examinable pattern is the shape, cheap object storage plus per-query SQL beats a cluster you pay for whether or not anyone queries, never the specific percentage.

Case C, EOS Group: DMS into S3 into Redshift, with PII masked in place

"By removing the need for on-premises infrastructure, EOS Group achieved a 50 percent reduction in infrastructure costs …": AWS EOS Group case study (accessed Oct 2026)

Read EOS as the migration sequence the exam keeps testing: DMS lands the database in S3, Redshift loads from S3, and masking happens inside the warehouse against the user's permissions: one table, two views of it, no second masked copy to keep in sync. Three traps hide here: DMS is not DataSync (it moves schemas and rows, not files), Macie cannot mask anything (it discovers sensitive data in Amazon S3), and the 50% is EOS's own outcome on their baseline, never a migration guarantee.

Case D, GE Aerospace: modernising a legacy operational data store

"We were seeing queries that used to run for an hour and a half running in 7 minutes using Amazon Redshift.": Bejoy John, Senior Director of Data Analytics, GE Aerospace (AWS case study, accessed Oct 2026)

GE Aerospace is the right-sizing story behind RA3 and managed storage: compute and storage are separate, so the fix for an hour-long query is a better-shaped warehouse rather than a larger copy of the same design. Exam-safe framing: 70% and >$500,000/year are GE's estimate against GE's baseline: the transferable fact is the selection (a curated, BI-shaped workload belongs in Redshift), not the number.

• 📚 Did you know? The governance cases read like the same number twice: Integral Ad Science (IAS) used Lake Formation tag-based access controls to reduce hundreds of permission rules down to precisely two, while GoDaddy's hub-and-spoke data mesh now holds over 2,000 data products across multiple petabytes in hundreds of accounts (AWS Big Data Blog, 2021-09-23 and 2022-11-21, accessed Oct 2026). Neither figure generalises to your workload, but both stems collapse to one selection rule: when the requirement is centrally governed access to data you must not copy, the answer is Lake Formation over Amazon S3 with Athena or EMR on top, never a second database standing in for a permission model.

---

Practice Questions

Comparative Verdict: how this topic compares on exam day

• Versus another cloud: DEA-C01 tests AWS services only: no item asks you to compare DynamoDB or Redshift with a competitor's offering. Answer with an AWS-documented behaviour (400 KB items, 3,500/5,500 per prefix, $5.00/TB scanned, Iceberg snapshots); an option resting on an unverified third-party benchmark is out of scope by construction.
• Versus self-managed / on-premises: running your own Hadoop, cache or search cluster means patching, capacity guessing and idle hardware. The exam's preference is managed and priced for consumption, S3 for raw scale, DynamoDB for key access, Athena for pay-per-query, while still respecting the counter-case: if the stated requirement is a self-managed engine with specific hardware, that requirement wins. Never claim managed is always right; claim it is right for the stated pattern.
• Versus another AWS service: Redshift when the query is repeated, concurrent BI over curated tables; Athena when it is ad-hoc SQL over files in S3; Spectrum when the data must stay in S3 but be queried with warehouse SQL; OpenSearch for relevance and log drill-down; Glue + Lake Formation for the catalog and governance above them. Wrong answers almost always size the engine to habit rather than to the query shape.
• Versus a manual, human process: partition registration by hand, consistency-verification passes, and manual small-file cleanup all lose to Iceberg snapshots with hidden partitioning, strongly consistent S3, and automated compaction: choose the option with the least unmanaged manual effort, but do not confuse "managed" with "free": Athena still bills bytes scanned, and S3 storage classes still bill for what you keep.