Streaming Ingestion: Kinesis, Amazon MSK, and Landing Streams in the Lake

Lessons 02 through 05 put files, formats and tables on Amazon S3. Everything so far has been moved in batches. This lesson is about the other half of ingestion: events that arrive continuously and must land in the lake without losing their order, duplicating silently, or quietly costing more than the analytics team earns back. By the end you will be able to size a Kinesis stream from a records-per-second number, explain exactly where ordering stops, decide when a hot key is a partition-key problem rather than a capacity problem, configure a Firehose buffer that triggers on the right condition, and defend a bronze landing that is idempotent even though the pipeline is at-least-once.

In this lesson you will:

• state the shard contract, 1 MB/s write and 1,000 records/s in, 2 MB/s read and 5 read transactions/s out, per shard, in both capacity modes, and know what did not change when records grew to 10 MiB;
• choose between Provisioned, On-demand Standard and On-demand Advantage, including the 25 MB/s account-level floor that makes Advantage the wrong answer at low volume;
• trace a partition key through MD5 to a hash-key range to a shard, and explain why ordering is a per-key promise rather than a stream-wide one;
• detect and fix a hot key when on-demand auto-scaling cannot help you;
• size shared fan-out versus enhanced fan-out and price the difference;
• work with the KPL (collection, aggregation, rate limiting) and the KCL (leases, checkpoints, parentShardId), and know the KCL 1.x end-of-support date;
• configure Amazon Data Firehose end to end: buffer size and interval, Lambda transformation batches, JSON to Parquet conversion against a Glue schema, dynamic partitioning limits and error prefixes;
• land streaming data with the three bronze land patterns and attach the right delivery-semantics label to each arrow;
• run the at-least-once to idempotent MERGE pattern that makes plain-file bronze safe to promote;
• route device telemetry through AWS IoT Core rules, and know what Kinesis Data Video Streams is for;
• compare Amazon MSK provisioned, MSK Serverless, MSK Connect and MSK streaming tables, with the broker-versus-serverless break-even arithmetic;
• study two AWS customer cases (Cisco Webex on MSK, VMware Carbon Black on Kinesis) and a sourced October 2026 update box;
• practise with 12 exam-style questions plus four interactive checks.

---

1. The service map: five paths from event to lake

Every streaming design on AWS is a composition of the same five pieces. Confusing them is the single most common architecture error in this space, because two of them look like pipes and only one of them is a log you can replay.

• 📚 Did you know? The Firehose name in front of you is cosmetic but load-bearing for older material: AWS renamed Amazon Kinesis Data Firehose to Amazon Data Firehose on 9 February 2024, and states that there are "no other changes, including service endpoints, APIs, the AWS Command Line Interface (AWS CLI), the AWS Identity and Access Management (IAM) access policies, and Amazon CloudWatch metrics" (AWS What's New, 2024-02-09; Firehose document history, accessed 2026-10). So IAM actions are still firehose:*, and an old blog saying "Kinesis Data Firehose" is describing exactly the same service you are configuring today.

---

2. Kinesis Data Streams: the shard contract

2.1 One number governs everything

A shard is "a uniquely identified sequence of data records in a stream" and "a base throughput unit of a Kinesis data stream." The contract, identical in provisioned and on-demand capacity modes, is:

Two things are true at once and students routinely collapse them into one: bytes and record-count are independent caps. A stream of 400-byte records hits the 1,000 records/s wall at only 0.38 MB/s, so records-per-second binds long before bytes do. Conversely, 2 KB records at 1,000/s use only 2 MB/s of the read budget but exactly the full write record rate.

The 2025 record-size change is the trap in this section. On 28 October 2025 AWS raised the maximum record size "up to 10MiB, a tenfold increase from the previous limit" and doubled the PutRecords request ceiling to 10 MiB, while stating plainly that "Individual shard throughput limits of 1MiB/s for writes and 2MiB/s for reads remain unchanged", large records "temporarily burst up to 10MiB/s, eventually averaging out to 1MiB per second." AWS recommends keeping large records at "1-2% of your total record count for optimal performance." The concepts page still carries the older "data blob can be up to 1 MB" sentence (accessed 2026-10); the limits page and the October 2025 announcement are the current facts.

• 📚 Did you know? PutRecords is deliberately not atomic: the AWS Big Data Blog records that "PutRecords always returns a 200 result, even if some records within the batch have failed" and that "A single record failure does not stop the processing of subsequent records." So a 200 OK from PutRecords means the request was accepted, not your batch landed, you must inspect FailedRecordCount and per-record ErrorCode values. On a single host, a raw PutRecord loop measured about 5,500 RPS with batched PutRecords versus roughly 76,000 RPS through the KPL, "about 800 times greater" (AWS Big Data Blog, 2015-08-07).

2.2 Capacity modes: three shapes, one contract

Practical rules that fall out of that table:

• You can switch modes twice per 24 hours per stream: a guardrail, not a free toggle. Design the switch as an event, not a loop.
• On-demand is not "no limits." A new on-demand stream starts at 4/8 MB/s, non-flagship Regions top out at 200/400 MB/s, and, the important one, on-demand "automatically adapts to handle uneven data distribution patterns unless a single partition key exceeds a shard's 1 MB/s throughput and 1000 records per second limits."
• Advantage is cheaper per GB but has a floor. The announcement is explicit: the account "commits to streaming with at least 25 MiB/s of data ingest and 25 MiB/s of data retrieval usage", in exchange for data ingested priced 60% lower, retrieval 60% lower, enhanced fan-out retrieval 68% lower and extended retention 77% lower (AWS Big Data Blog, 2025-11-03). A 0.5 MB/s development stream will pay for 25 MB/s it does not use.

2.3 Retention: from replay window to long-term archive

Default retention is 24 hours, and it is the minimum; IncreaseStreamRetentionPeriod accepts up to 8,760 hours (365 days). Three cost tiers sit behind that knob:

Replay is the reason retention exists: Kinesis "provides ordering of records, as well as the ability to read and/or replay records in the same order to multiple Amazon Kinesis Applications." A 24-hour window means a consumer bug discovered at 09:00 can be repaired and reprocessed from 09:00 the previous day, and nothing more. If your bronze-on-S3 landing is the durable copy (lesson 05), the stream's retention is a reprocessing budget, not your backup strategy.

---

3. Partition keys, ordering, and hot shards

3.1 The path a record actually takes

The producer supplies a partition key: "Partition keys are Unicode strings, with a maximum length limit of 256 characters for each key." AWS then uses "an MD5 hash function … to map partition keys to 128-bit integer values and to map associated data records to shards using the hash key ranges of the shards." All records sharing a key land in the same shard, which is why "the Kinesis Client Library (KCL) delivers all records for a given partition key to the same record processor."

Ordering, stated precisely, has exactly three scopes and no more:

1. Within a shard: "records that Kinesis Data Streams receives through a GetRecords call are strictly ordered by sequence number."
2. Within a partition key: the key always maps to one shard, so key-local order survives.
3. Same client, same key, opt-in: SequenceNumberForOrdering "Guarantees strictly increasing sequence numbers, for puts from the same client and to the same partition key … If this parameter is not set, records are coarsely ordered based on arrival time."

There is no stream-wide order, and PutRecords does not even order inside its own batch. If your downstream MERGE assumes a global sequence, you have designed a guarantee the service does not provide.

3.2 Worked example E1, records/s binds first: sizing and cost

Scenario (as of Oct 2026): 2,500 records/s at 400 bytes average, steady 24x7, US East (N. Virginia).

Note the two rounding rules that make streaming bills surprising: Kinesis data-in rounds each record up to the nearest 1 KB ("if your data records are 4.5 KB each, Kinesis Data Streams will count each record as 5 KB"), while the provisioned PUT meter bills in 25 KB payload units. A pipeline of 900-byte records is billed as 1 KB per record on on-demand and as one unit per record on provisioned, and it is records/s that sizes your shards either way.

3.3 Worked example E2: the hot key that auto-scaling cannot fix

Scenario: one tenant_id = acme key sends 1,500 records/s of 80 bytes.

AWS's own remedies: "use the split-shard operation … If locality or order based on the current partition key is not required, change the partition key scheme to increase its distribution. If the producer application is exceeding the throughput limit of a shard due to a single partition key (hot key issue), change the partition key scheme." Advanced producers can bypass the MD5 mapping with ExplicitHashKey to place a key deliberately: useful for draining a hot shard, dangerous if you use it to paper over a key design that should change.

• 📚 Did you know? A useful production heuristic comes from Chime Financial's fraud platform: AWS's write-up of their streaming stack notes "we land on a 1:1 mapping between Kinesis Shard and vCPU core" while ingesting "more than 1 million events per second" with end-to-end latency "less than 15 seconds" (AWS Big Data Blog, 2023-09-19). Treat it as one team's worker-sizing rule of thumb for a KCL fleet, not as a service quota: worker count must still be at least shard count for parallelism, and every lease lives in DynamoDB.

---

4. Fan-out: shared reads versus enhanced fan-out

A stream is only useful if several consumers can read it. Kinesis gives you two mechanisms with genuinely different economics.

The FAQ sentence that matters: "The shared fan-out consumers all share a shard's 2 MB/second of read throughput and five transactions per second limits … An enhanced fan-out consumer gets its own 2 MB/second allotment of read throughput." Add a second polling consumer and each one's effective read rate halves: the classic production symptom where "adding a consumer made everything slower."

4.1 Worked example E3: fan-out sizing and the EFO bill

Scenario: a 4-shard stream with three readers: (a) a Lambda poller using the shared path, (b) a Flink application needing low latency, (c) Firehose reading Kinesis Data Streams.

Firehose's own FAQ behaviour is what forces the decision: "Firehose calls Kinesis Data Streams GetRecords() once every second for each Kinesis shard." If Firehose is your sink, assume the shared pool is spoken for.

---

5. Producers and consumers: KPL, KCL, and the lease table

5.1 KPL: collection, aggregation, rate limiting

The Kinesis Producer Library runs as "a separate process, and communicates with your parent user process using IPC", and does four jobs: automatic retries, collection (batching many records into PutRecords requests), aggregation (packing many user records into one Kinesis record), and per-shard rate limiting with a token bucket "by default … set 50 percent higher than the actual shard limit." It also publishes CloudWatch metrics on your behalf and adds latency up to RecordMaxBufferedTime.

Worked example E4: aggregation defeats the records/s cap.

AWS's own example matches: "With KPL aggregation, you can pack 1,000 records into only 10 Kinesis Data Streams records, reducing the RPS to 10 (at 50 KB each)", and "The Kinesis Data Streams records per second limit binds customers with records smaller than 1 KB."

Portability is the tax. Per AWS's usage matrix, a producer in any language other than Java cannot use the KPL, and a Java KPL producer feeding a non-Java consumer must disable aggregation: otherwise the consumer has to de-aggregate records it never expected. Raw PutRecords is portable; you own batching, partial-failure handling, retries and backoff.

5.2 KCL: leases, checkpoints, and parentShardId

The Kinesis Client Library is compiled into your consumer and gives you:

• Leases: "Each shard is bound to only one worker by a lease at any given time", tracked in a DynamoDB lease table. KCL 3.x creates three tables by default (lease, worker metrics, coordinator state); from KCL 3.5 the single-table format consolidates all metadata into the lease table.
• Checkpointing: "the process of persistently storing the position of the last successfully processed record in a shard", stored in that lease table. Restart from the checkpoint, not from "now".
• parentShardId: "Used to ensure that the parent shard is fully processed before processing starts on the child shards. This ensures that records are processed in the same order they were put into the stream." This is the only mechanism that keeps ordering across a reshard event.

Two dates matter here. KCL 1.x reached end-of-support on 30 January 2026, and AWS's guidance is to "migrate your KCL applications using version 1.x to the latest KCL version": as of this lesson that date has passed, so any 1.x consumer is running on borrowed time. And KCL is not free infrastructure: DynamoDB tables (on-demand capacity recommended) and CloudWatch metrics ride along, and worker count ≥ shard count is a parallelism requirement, not a tuning preference.

---

6. Amazon Data Firehose: a delivery service, not a processor

6.1 The pipeline, stage by stage

Firehose is "a fully managed service for delivering real-time streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon OpenSearch Service, Amazon OpenSearch Serverless, Splunk, Apache Iceberg Tables, and any custom HTTP endpoint." Its stages, in order:

1. Source: Direct PUT, Kinesis Data Streams, Amazon MSK, CloudWatch Logs, EventBridge, IoT, WAF, Network Firewall, HTTP.
2. Buffer: "Amazon Data Firehose buffers incoming streaming data to a certain size or for a certain period of time before delivering it to destinations"; whichever condition is met first fires the delivery.
3. Lambda transformation (optional): per-batch invoke, payload capped at 6 MB synchronous, invocation up to 5 minutes, default 3 retries (configurable 0 to 300).
4. Record format conversion (optional): JSON to Parquet or ORC using a schema in the AWS Glue Data Catalog (deserializer, schema, serializer).
5. Dynamic partitioning (optional): JQ-based keying into S3 prefixes, up to 500 active partitions (raisable to 2,500).
6. Destination, S3, Iceberg tables, Redshift (via S3 + COPY), OpenSearch, Splunk, Snowflake, HTTP, plus error and backup prefixes.

6.2 Worked example E5; which buffer condition fires first

Buffering hints for an S3 destination: size 1 to 128 MB (default 5), interval 0 to 900 s (default 300).

Three modifiers change the arithmetic:

• Record format conversion raises the floor. With conversion enabled you "can't set SizeInMBs to a value less than 64", and "The value becomes 128 when you enable it."
• Dynamic partitioning costs latency. "Firehose uses multi-stage buffering internally. Therefore, end-to-end delay of a batch of records might be 1.5 times of the configured buffering hint time", and zero buffering is unavailable with the feature.
• Sub-60-second buffering changes the S3 bill. "Firehose uses multi-part upload for S3 destination when you configure a buffer time interval less than 60 seconds … you will see some increase in S3 PUT API costs."

• 📚 Did you know? Firehose's quotas page quietly contains the scaling argument for Kinesis-as-source: Direct PUT into Firehose is capped (in us-east-1/us-west-2/eu-west-1) at "500,000 records/second, 2,000 requests/second, and 5 MiB/second", but "When Kinesis Data Streams is configured as the data source, this quota doesn't apply, and Amazon Data Firehose scales up and down with no limit." And because Firehose reads "once every second for each Kinesis shard", the two services share your shard sizing: more shards means more parallel Firehose reads, not just more write capacity.

---

7. Landing in the lake with Firehose

7.1 What Firehose actually promises

The delivery semantics differ per destination, and this is the most misquoted area in the whole lesson:

The design consequence: anything that lands as plain files in a bronze prefix is at-least-once. Duplicates are not a bug you can configure away; they are the contract. You make the table correct downstream (Section 9.3), not the delivery perfect.

7.2 Worked example E6: a Firehose invoice for a JSON to Parquet lake landing

Scenario: 100 records/s x 3 KB, JSON converted to Parquet, delivered to S3, US East (N. Virginia), 30 days.

The teaching point is the shape of the bill, not the cents: ingestion is the smallest lever when you switch features on. Conversion and dynamic partitioning together more than double the base $35.84, and each of them is a feature you chose in exchange for queryable Parquet on partitioned prefixes instead of a pile of JSON. Note also the source-dependent rates: a Kinesis Data Streams source bills $0.029/GB, an Amazon MSK source bills $0.055/GB with no 5 KB rounding, vended logs $0.13/GB, an Iceberg destination (Kinesis source) $0.045/GB, and VPC delivery $0.01/GB plus $0.01 per AZ-hour.

7.3 Kinesis data delivery and streaming tables (no Firehose required)

As of October 2026 a Kinesis stream can land in the lake without Firehose at all. Data delivery supports "two destination types: streaming tables on Apache Iceberg, and general purpose Amazon S3 buckets", with "Built-in intelligent inline compaction [that] eliminates small file proliferation." Requirements and prices:

AWS's stated outcomes for streaming tables: "reduce data delivery costs to S3 Tables by up to 50% compared to self-managed alternatives and reduce downstream query costs by up to 30% through intelligent inline compaction", with data "queryable within minutes of being published to your stream." Provisioned mode does not support data delivery or streaming tables at all: a fact that removes provisioned mode from consideration for this pattern.

---

8. AWS IoT Core rules and Kinesis Data Video Streams

8.1 MQTT, rule, action, lake

AWS IoT Core is the ingress for device telemetry. A rule evaluates an MQTT message with AWS IoT SQL and invokes one or more actions. The documented action list includes kinesis (Kinesis Data Streams), firehose, s3, timestream, lambda, sqs, sns, dynamodb, kafka (Apache Kafka), stepFunctions, openSearch, republish, cloudWatch, location, influxDB and iotSiteWise.

The Kinesis action takes three parameters you must get right: the stream, the partitionKey, "usually composed of an expression (for example, ${topic()} or ${timestamp()})", and the roleArn, "An IAM role that AWS IoT can assume to perform the kinesis:PutRecord operation." Choosing ${topic()} means all messages on one MQTT topic hash to one shard: elegant for ordering, a hot-key generator at volume. Choosing a device- or event-level key spreads load and drops topic ordering.

One cost control is worth memorising: "You can also use the Basic Ingest feature to securely send device data to the AWS services listed previously, without incurring messaging costs. The Basic Ingest feature optimizes data flow by removing the publish/subscribe message broker from the ingestion path." If the data never needs MQTT fan-out, do not pay for the broker hop.

8.2 Kinesis Data Video Streams: the media lane, briefly

Amazon Kinesis Video Streams carries "media from connected devices to AWS for storage, analytics, machine learning (ML), playback, and other processing", with managed HLS and DASH playback, WebRTC for ultra-low-latency two-way streaming, and integrations with Rekognition Video and SageMaker. It is the correct answer for cameras, audio and time-serialized binary: not for JSON event records.

US East pricing (as of Oct 2026): $0.0085/GB ingested, $0.0085/GB consumed (HLS consumption $0.0119/GB), storage $0.023/GB-month on the hot tier; WebRTC adds $0.03 per month per active signaling channel, $2.25 per million signaling messages and $0.12 per thousand TURN streaming minutes. A warm storage tier (feature dated 26 November 2025) offers ingest at $0.0100 per 1,000 persisted fragments and storage at $0.0125/GB-month with a 30-day minimum retention: the video equivalent of Kinesis stream retention tuning.

• 📚 Did you know? The cheapest IoT rule is often the one you delete. Rules with wildcards can match most messages, and each action assumes an IAM role and bills the target service, so a single broad rule fanning to Kinesis and Firehose and S3 is three bills for one message. Design the rule's SQL as narrowly as you would a database index, and reach for Basic Ingest when the broker adds nothing.

---

9. Amazon MSK: managed Kafka, and getting from a stream to a table

9.1 Provisioned versus Serverless

AWS's own broker-size guidance: "M7g brokers use AWS Graviton processors … M7g brokers offer improved price performance relative to comparable M5 instances" and "M7g and M5 brokers are recommended for production workloads." The break-even logic is the exercise: provisioned has a fixed floor (three brokers' worth of hours, plus storage you pay for whether traffic arrives or not), while serverless meters partitions and bytes, so a steady high-volume stream can cost more on Serverless than on provisioned brokers, and the 200 MBps ingress ceiling is a hard stop, not a warning.

Worked example E8: the break-even shape (AWS's published examples, as of Oct 2026).

9.2 MSK Connect and stream-to-lake sinks

MSK Connect "makes it easy for developers to stream data to and from their Apache Kafka clusters … uses Kafka Connect versions 2.7.1 or 3.7.x", can "deploy connectors developed by 3rd parties like Debezium … or deploy an existing connector with no code changes", and "Connectors automatically scale to adjust for changes in load and you pay only for the resources that you use", with automatic task restarts and PrivateLink for private connectivity.

The sink paths that matter for this course:

• S3 sink (Confluent connector on MSK Connect): "connector.class": "io.confluent.connect.s3.S3SinkConnector", with "flush.size": "1" controlling object size and provisioned capacity expressed as mcuCount / workerCount.
• Iceberg sink: the AWS-published real-time CDC lake recipe is MSK Connect + Iceberg Kafka Connect (AWS Big Data Blog, 2026-02-03), where the connector's iceberg.control.commit.interval-ms sets how often stream offsets become table snapshots.
• Firehose with MSK as source: console path: "For Source, choose Amazon MSK. For Destination, choose Apache Iceberg Tables." Note the price: MSK-source ingestion is $0.055/GB versus $0.029/GB for a Kinesis source.
• MSK streaming tables, Kafka topics materialized as Iceberg on S3 Tables: "Amazon MSK Express brokers write your records as Apache Parquet data files and commit them to the table", scaling to "10 GBps of delivery throughput" at $10.00 per TB delivered, freshness configured between 5 and 15 minutes, with a required dead-letter queue bucket and the explicit caveat that "A Channel does not backfill previously produced data, only data produced after enablement is delivered."

For lab work, the awslabs MSK Data Generator is a Kafka Connect source connector (genkp.<topic>.with for the partition key, genv.<field>.with for values) deployable on MSK Connect or in an ECS container: the streaming equivalent of a seed script.

9.3 Worked example E7: at-least-once bronze to idempotent silver

Scenario: Firehose delivered a 10-minute window of JSON-to-Parquet files to s3://lake/bronze/events/dt=2026-10-11/hr=09/, and a retry duplicated 1,412 of them. Promote to a silver Iceberg table.

Rules that travel with this pattern: Iceberg MERGE accepts only one source record per target row: dedupe first, always; and the gate s.event_ts > t.event_ts makes a replayed older event a no-op instead of a downgrade. The alternative is to skip plain files entirely and use an exactly-once destination (Firehose to Iceberg, or streaming tables), where write coordination handles the duplicates for you.

The semantics ledger, stated once and never blurred:

• Plain files in bronze = at-least-once. Plan dedupe (deterministic event id plus commit window) or MERGE on a surrogate id with a monotonically increasing source timestamp.
• Documented exactly-once exists, but scoped: Firehose's Iceberg destination ("Firehose guarantees exactly-once delivery to Iceberg tables"), Kinesis streaming tables ("exactly once delivery semantics across all shards"), and Flink's exactly-once processing.
• "Exactly-once" in a vendor slide is not the same claim as "exactly-once in the documentation." Ask which destination, which catalog, and what happens on a delivery timeout.

---

10. Choosing: the decision matrix

---

2026 Updates (as of October 2026)

---

Real-World Case Studies

Case A, Cisco Webex: 12 MSK clusters and the on-call you get back

Cisco (Webex observability platform). AWS's case study describes a self-managed Kafka estate for the observability/ELK stack whose operational burden fell on a team over-provisioning "one-size-fits-all instances" while outages hit thousands of internal users. The migration itself was scoped honestly: "The observability team spent 5 months engineering the migration from self-managed Kafka to Amazon MSK and then implemented the new stack in 6 weeks."

The published steady state is the number that makes this lesson's MSK section concrete: "The entire system is composed of 12 Amazon MSK clusters with a total of 66 brokers. The system seamlessly handles a daily throughput of 147 TB of data and more than 100 billion messages." Downstream effects: "More than 4,000 Webex developers, managers, and team members … have benefited", and "reduced the data transport time from the backend to the user, so retrieval times are under 10 seconds." AWS's summary of the outcome: "Cisco has reduced its operational overhead to virtually zero with cost savings and decreased service outages." Robert Labrie, manager for observability at Webex: "We run a globally federated observability stack at scale, and it just works."

Lesson angle: managed Kafka converts a staffing problem into a capacity problem. The protocol, the clients and the Connect ecosystem stayed; what disappeared was broker babysitting, and the 5-month migration line is the honest price of that trade. 66 brokers across 12 clusters is also the scale at which Section 9.1's broker-hour arithmetic stops being a spreadsheet curiosity.

Customer-reported figures from an AWS case-study page (undated, accessed 2026-10-11); not an AWS commitment.

Case B, VMware Carbon Black: 1 PB per day and shard elasticity as the cost lever

VMware Carbon Black (endpoint security). From AWS's Kinesis Data Streams customer page: "VMware Carbon Black analyzes over a trillion events every day … Now they process 1 PB streaming data per day with reduced operating and infrastructure costs." The architecture is ingesting from "millions of distributed edge devices" into Amazon Kinesis Data Streams, with real-time analytics in milliseconds.

The quote that explains why Kinesis rather than a broker fleet is Stoyan Dimkov, Staff Engineer and Software Architect: "When individual customers' data increases or decreases, we can use the elasticity of Amazon Kinesis Data Streams to scale compute up or down to process data reliably while effectively managing our cost."

Lesson angle: at 1 PB/day the interesting number is not the shard count; it is that elasticity per customer workload is the cost model. Sizing that stream is E1's arithmetic at scale (1 PB/day is roughly 11.6 MB/s average before replicas, which on-demand absorbs without a shard spreadsheet), and the per-customer volume swings are exactly what makes provisioned shard planning painful.

Customer-reported figures from an AWS product customer page (undated, accessed 2026-10-11); not an AWS commitment.

Case C, TrueCar: schedule removal, not engine tuning

TrueCar (auto marketplace). On the same AWS Kinesis customer page: "By adopting a streaming-data strategy with Amazon Kinesis Data Streams, TrueCar has improved the speed to insight of clickstream analytics by 48 times, automated its extract, transform, load (ETL) process, and improved observability into its metrics." 48x is a latency story, not a throughput story: the batch ETL schedule was the bottleneck, and the stream removed it.

Customer-reported figures from an AWS product customer page (undated, accessed 2026-10-11); not an AWS commitment.

---

Practice Questions