Setup AWS MWAA in 7 Easy Steps
How to set up Airflow in a few mins vs hours or days.
Apr 12, 20225 min read302
Command Palette
Search for a command to run...
#big-data
Articles tagged with #big-data
Apache Airflow, Not Just for Data Pipelines
Alternate Use Cases
Sep 1, 20213 min read289Data Versioning Your Processed Data
Does your team or company track the versions of your processed data? If not, maybe one should consider the added value of data versioning. When I use the term "data versioning", what I mean is tracking how your data such as an entity has changed ove...
Jul 27, 20214 min read142Increase Data Processing Throughput of a Step Function with Dynamic Parallelism
Is your data growing beyond what a Lambda-based Step Function Lambda can handle? Are you having problems with AWS Step Functions timing out in 900 seconds because your Lambda function was too slow at processing data? Maybe your data is large but not ...
Jun 29, 20212 min read194