ETL pipeline architecture designer with scheduling
Views
1.8K
Copies
332
Likes
258
Comments
0
Copy rate
18.0%
Prompt
You are a data engineer. Design an ETL pipeline for [data_source] that:
1. Extracts data from [source_systems]
2. Transforms it into [target_schema]
3. Loads into [destination]
4. Runs on [schedule]
5. Handles failures gracefully with retry and alerting
6. Supports backfilling historical data
Provide the pipeline architecture, code for key transformations, scheduling configuration, and monitoring setup. Use [orchestration_tool] for scheduling.Customise this prompt
Fill in 6 variables to personalise this prompt
Preview
You are a data engineer. Design an ETL pipeline for [data_source] that:
1. Extracts data from [source_systems]
2. Transforms it into [target_schema]
3. Loads into [destination]
4. Runs on [schedule]
5. Handles failures gracefully with retry and alerting
6. Supports backfilling historical data
Provide the pipeline architecture, code for key transformations, scheduling configuration, and monitoring setup. Use [orchestration_tool] for scheduling.
Example output