An organization is designing an integration solution to replicate financial transaction data from a legacy system into a data warehouse (DWH).
The DWH must contain a daily snapshot of financial transactions, to be delivered as a CSV file. Daily transaction volume exceeds tens of millions of records, with significant spikes in volume during popular shopping periods.
What is the most appropriate integration style for an integration solution that meets the organization's current requirements?
The daily snapshot requirement and the large volume of financial transactions necessitate an efficient method for handling bulk data transformation and loading. Batch-triggered ETL (Extract, Transform, Load) is the most appropriate integration style in this scenario. It allows for the aggregation and processing of large datasets at periodic intervals, which is suitable for creating daily CSV snapshots for the data warehouse. Additionally, batch processing can efficiently handle high volumes and spikes in data, ensuring that daily snapshots are delivered as required.
It should be B. D does not make sense. Microservice architecture has nothing to do with CSV file with millions of record. For file with millions of record, the only approach that will work is batch triggered.
B id correct answer
B most likely should be the correct answer
For large volume data we should use batch to process it faster. B is the correct ans.
B. Batch-triggered ETL
Here nothing is mentioned about interfacing with the legacy system. Is this legacy system emitting one or multiple events at a time? Or does a process need to pull some data from DB or file? How are you going to write a file in batch processing? In aggregator? Does file write operation support concurrency? The question doesn't provide all information and hence is confusing.
B is correct because of volume
B should be correct answer
B. Batch-triggered ETL
B is the correct answer
DWH must have a daily snapshot, typically achieve through batch processing. Gathering the transactions in a (near) real-time through EDA or micro service architecture would require changing the legacy source system.