Least Operational Overhead for Kinesis to Redshift Ingestion

DEA-C01 By islever Editorial Last updated

Perform data ingestion. Automate data processing by using AWS services.

Short answer

D Connect Kinesis Data Streams to Amazon Kinesis Data Firehose, stage data in Amazon S3, and use the COPY command to load data into Amazon Redshift for least operational overhead.

Question

A company wants to implement real-time analytics capabilities. The company wants to use Amazon Kinesis Data Streams and Amazon Redshift to ingest and process streaming data at the rate of several gigabytes per second. The company wants to derive near real-time insights by using existing business intelligence (BI) and analytics tools.
Which solution will meet these requirements with the LEAST operational overhead?
  1. Use Kinesis Data Streams to stage data in Amazon S3. Use the COPY command to load data from Amazon S3 directly into Amazon Redshift to make the data immediately available for real-time analysis.
  2. Access the data from Kinesis Data Streams by using SQL queries. Create materialized views directly on top of the stream. Refresh the materialized views regularly to query the most recent stream data.
  3. Create an external schema in Amazon Redshift to map the data from Kinesis Data Streams to an Amazon Redshift object. Create a materialized view to read data from the stream. Set the materialized view to auto refresh.
  4. Connect Kinesis Data Streams to Amazon Kinesis Data Firehose. Use Kinesis Data Firehose to stage the data in Amazon S3. Use the COPY command to load the data from Amazon S3 to a table in Amazon Redshift. Correct Answer

Key Takeaways

The main trap is assuming Redshift can query Kinesis directly; using Firehose as a managed intermediary to S3 reduces operational overhead significantly.

Learn how to ingest streaming data from Amazon Kinesis Data Streams into Amazon Redshift with minimal management using Kinesis Data Firehose and S3 staging.

Common mistake: Choosing Option C because it mentions 'auto-refresh' and 'materialized views,' but this fails because Redshift cannot create external schemas directly mapping to Kinesis Data Streams without an intermediary like Firehose or S3.

Expert Analysis

The scenario requires ingesting streaming data from Amazon Kinesis Data Streams into Amazon Redshift for real-time analytics with the LEAST operational overhead. The key constraint is minimizing management while handling high-throughput data (several GB/s).

Option D leverages Amazon Kinesis Data Firehose, a fully managed service designed specifically to capture, transform, and load streaming data to destinations like Amazon S3 and Amazon Redshift. By connecting Kinesis Data Streams to Firehose, the company offloads buffering, scaling, error handling, and delivery logic to AWS. Firehose then stages the data in Amazon S3 (a durable, cost-effective storage layer) and uses the COPY command to load it into Redshift. This pattern is the standard, lowest-effort approach for batch or near-real-time ingestion into Redshift because it eliminates the need for custom code or complex schema management.

Option C suggests creating an external schema in Redshift to map directly to Kinesis Data Streams and using auto-refreshing materialized views. However, as noted in community comments, Amazon Redshift does not support creating external schemas that map directly to Kinesis Data Streams. You typically need an intermediary service like Firehose to land data in S3 or DynamoDB before Redshift can query it via external tables or materialized views. Even if direct integration were possible, managing auto-refresh intervals adds operational complexity compared to Firehose's automated delivery.

Option A proposes staging data in S3 but implies a manual or custom mechanism to move data from Kinesis to S3 without Firehose, which increases operational overhead. Option B suggests querying Kinesis directly via SQL, which is not a supported capability for Redshift users; Redshift queries structured data in tables or external sources, not live streams directly.

Therefore, Option D provides the most robust, fully managed pipeline with minimal operational overhead by utilizing Firehose as the bridge between Kinesis and S3/Redshift.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When asked for 'least operational overhead' in data ingestion scenarios, prioritize fully managed services like Kinesis Data Firehose over custom pipelines or complex configurations. Always verify if the target database supports direct connections to the source; often, an intermediate storage layer (like S3) is required and handled by a managed service to minimize maintenance.

Frequently Asked Questions

Can Amazon Redshift query Amazon Kinesis Data Streams directly?

No, Amazon Redshift cannot create external schemas that map directly to Kinesis Data Streams. You must use an intermediary service like Kinesis Data Firehose to deliver data to Amazon S3 or DynamoDB first.

Why is Kinesis Data Firehose preferred for Redshift ingestion?

Firehose is fully managed, automatically handles scaling, buffering, and error recovery, and integrates seamlessly with S3 and Redshift via the COPY command, reducing operational overhead compared to custom scripts.

More DEA-C01 FAQ →

Community Votes

C
46%
D
46%
A
8%

46% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 34 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 30 of 34

  1. blackgamer 👍 11 Answered C
    The answer is C. It can provide near real-time insight analysis. Refer the article from AWS - https://aws.amazon.com/blogs/big-data/real-time-analytics-with-amazon-redshift-streaming-ingestion/
  2. helpaws 👍 8 Answered C
    Key word here is near real-time. If it's involve S3 and COPY, it's not gonna be near real-time
  3. markill123 👍 7
    Redshift cannot create external schemas that map directly to Kinesis Data Streams. You would still need an intermediary step, such as Firehose or S3, to handle data ingestion. Additionally, maintaining auto-refreshing materialized views directly from a stream isn't feasible with Redshift.
  4. BartoszGolebiowski24 👍 6
    However, after some investigation, I found out that Amazon Kinesis Data Streams provides a way to ingest stream data directly into an Amazon Redshift cluster. https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-streaming-ingestion.html Materialized views with auto-refresh enabled will continuously ingest new data from the stream as it arrives, keeping the view updated with the latest data in real time. So I think the correct answer is C. The COPY command also supports loading data from streaming sources like Kinesis Data Streams or Kinesis Data Firehose. When used with these services, COPY provides a way to ingest real-time streaming data into Redshift tables. But this solution is not an option for this question.
  5. fceb2c1 👍 5 Answered C
    https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-streaming-ingestion-getting-started.html C is correct. (KDS -> Redshift) D is wrong as it has more operational overhead (KDS -> KDF -> S3 -> Redshift)
Show 25 more comments
  1. markill123 👍 4 Answered D
    Here’s why D is the best choice: Kinesis Data Firehose is a fully managed service that automatically handles the ingestion of data from Kinesis Data Streams and stages it in S3, which significantly reduces operational overhead compared to managing custom data ingestion pipelines. S3 as a staging area: Using Amazon S3 as a staging location allows for flexible data management, high durability, and direct loading into Redshift without needing to manage complex buffering or data handling processes. COPY command: The COPY command in Amazon Redshift is highly optimized for loading large datasets efficiently, making it a common and effective method to load bulk data from S3 into Redshift for near real-time analysis. Firehose to Redshift: Firehose can automatically buffer, batch, and transform data before loading it into Redshift, reducing manual intervention and ensuring data is readily available for real-time analytics.
  2. Rpathak4 👍 3 Answered D
    ✅ Use Kinesis Data Firehose to load data into Redshift via S3 for the simplest and most scalable solution. ✅ Firehose automatically batches, transforms, and loads data with no manual intervention required. ✅ Achieves near real-time analytics with minimal operational effort.
  3. Christina666 👍 3 Answered C
    Using materialized views with auto-refresh directly on a Redshift external schema of Kinesis Data Stream offers the most streamlined and efficient approach for near real-time insights using existing BI tools.
  4. Asen_Cat 👍 3 Answered D
    D could be the most standard way to handle this case. How to use C to implement it is questionable for me.
  5. GiorgioGss 👍 3 Answered D
    I think D. It could be C but because of "LEAST operational overhead" I will go with D.
  6. certplan 👍 2
    1. Amazon Kinesis Data Firehose: It's designed to reliably load streaming data into data lakes and data stores with minimal configuration and management overhead. It handles tasks like buffering, scaling, and delivering data to destinations like Amazon S3 and Amazon Redshift automatically. 2. Amazon S3 as a staging area: Storing data in Amazon S3 provides a scalable and durable solution for data storage without needing to manage infrastructure. It also allows for easy integration with other AWS services and existing BI and analytics tools. 3. Amazon Redshift: While Redshift requires some setup and management, loading data from Amazon S3 using the COPY command is a straightforward process. Once data is loaded into Redshift, existing BI and analytics tools can query the data directly, enabling near real-time insights. 4. Minimal operational overhead: This solution minimizes operational overhead because much of the management tasks, such as scaling, buffering, and delivery of data, are handled by Amazon Kinesis Data Firehose. Additionally, using Amazon S3 as a staging area simplifies data storage and integration with other services.
  7. plutonash 👍 2 Answered D
    A: Kinesis Data Streams to stage data in Amazon S3. not really easy, B: sql directly to Kinesis Data Streams : functionality not exist C : external schema from redshift to Kinesis Data Streams : functionality not exist D : near real-time = Kinesis Data Firehose
  8. shammous 👍 2 Answered D
    Option C has an issue: Redshift does not natively support direct querying or mapping of Kinesis Data Streams. D is the only correct option.
  9. d8945a1 👍 2 Answered C
    MVs in Redshift with auto refresh is the best option for near real time.
  10. V0811 👍 2 Answered D
    Option D
  11. certplan 👍 1
    Point: "Which solution will meet these requirements with the LEAST operational overhead?" C. - This approach involves creating an external schema in Amazon Redshift to map data from Kinesis Data Streams, which adds complexity compared to directly loading data from Amazon S3 using Amazon Kinesis Data Firehose. - While materialized views with auto-refresh can provide near real-time insights, managing them and ensuring proper synchronization with the streaming data source may require more operational effort. - AWS documentation for Amazon Redshift primarily focuses on traditional data loading methods and querying, with limited guidance on integrating with real-time data sources like Kinesis Data Streams.
  12. heavenlypearl 👍 1 Answered C
    Amazon Redshift can automatically refresh materialized views with up-to-date data from its base tables when materialized views are created with or altered to have the autorefresh option. Amazon Redshift autorefreshes materialized views as soon as possible after base tables changes. https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-refresh.html
  13. bakarys 👍 1 Answered A
    Option A (using Kinesis Data Streams to stage data in Amazon S3 and loading it directly into Amazon Redshift) is the most straightforward and efficient approach. It minimizes operational overhead and ensures immediate availability of data for analysis. Options B and C introduce additional complexity and may not provide the same level of efficiency
  14. certplan 👍 1
    By considering the characteristics and capabilities of each AWS service and approach, along with insights from AWS documentation, it becomes evident that option D offers the most streamlined and operationally efficient solution for the scenario described. This idea/concept is also straight out of the Amazon Solutions Architect course material.
  15. MephiboshethGumani 👍 1 Answered D
    Creating an external schema and using materialized views directly on top of Kinesis Data Streams is also not an ideal choice because this approach can add complexity and doesn't leverage fully managed solutions like Kinesis Data Firehose. The manual management of data refresh rates adds operational overhead.
  16. BartoszGolebiowski24 👍 1
    I think this is true. I could not find any sources that AWS Kinesis Data Stream can stream data directly into s3 without a middle step with AWS Kinesis Data Firehose. The AWS Kinesis Data Firehose is near real-time service. Anyway, I think the answer is D because the other 3 options are not better at all.
  17. haby 👍 1 Answered A
    A for me C - Redshift does not natively support direct mapping to Kinesis Data Streams. Some extra configs are needed. D - There will be a 60s latency when using Firehose, so it's "Near" real time not real time.
  18. melligeri 👍 1 Answered C
    https://aws.amazon.com/blogs/big-data/real-time-analytics-with-amazon-redshift-streaming-ingestion/#:~:text=Before%20the%20launch,the%20data%20stream.
  19. royalrum 👍 1
    Firehose is Near-Real time, you can set your buffer size and stream to either Redshift or S3 directly. Since Redshift is not in the option, use s3...
  20. HagarTheHorrible 👍 1 Answered D
    Redshift does not natively support direct mapping to Kinesis Data Streams. Materialized views cannot directly query streaming data from Kinesis.
  21. Deno_Dennis 👍 1 Answered C
    Review the explanation from AWS https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-streaming-ingestion-getting-started.html
  22. jesusmoh 👍 1 Answered D
    option D provides a streamlined, efficient, and low-overhead approach to achieving real-time analytics with the specified technologies.
  23. Shatheesh 👍 1 Answered D
    Kinesis Data Streams , option D using Kinesis Data Firehose is a fully managed service that automatically handles the ingestion of data
  24. Eltanany 👍 1 Answered C
    Refer to the article from AWS - https://aws.amazon.com/blogs/big-data/real-time-analytics-with-amazon-redshift-streaming-ingestion/
  25. altonh 👍 1 Answered C
    See https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-streaming-ingestion-getting-started.html

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Amazon Redshift Compound Sort Key Query Optimization

S-grade

Secure Redshift Spectrum Access with VPC Endpoints

S-grade

Athena Federated Query for Multi-Source Iceberg Pipelines

S-grade

Reclaiming Storage in Amazon Redshift Materialized Views

S-grade

Practice All DEA-C01 Questions

Access 100 questions with complete answers and detailed explanations.

View Full DEA-C01 Practice Test →

← Back to DEA-C01 Study Guide