Setup AWS MWAA in 7 Easy Steps
How to set up Airflow in a few mins vs hours or days.
Latest articles
Alley-oop
DE tosses up the data to DS for the Business Value Alley-oop!
Apache Airflow, Not Just for Data Pipelines
Alternate Use Cases
Data Versioning Your Processed Data
Does your team or company track the versions of your processed data? If not, maybe one should consider the added value of data versioning. When I use the term "data versioning", what I mean is tracking how your data such as an entity has changed ove...
Brickit: Computer Vision App that Shows You Lego Ideas.
Like many of us "nerds" out there, we probably have a bucket or three of Legos sitting somewhere. Most of my collectable "Adult" Legos get built once based on the set instructions. However, if you're looking to build something new and interesting wit...
Increase Data Processing Throughput of a Step Function with Dynamic Parallelism
Is your data growing beyond what a Lambda-based Step Function Lambda can handle? Are you having problems with AWS Step Functions timing out in 900 seconds because your Lambda function was too slow at processing data? Maybe your data is large but not ...
How to Not Lose Your Job to AI: Don't Be An Endpoint
The fear of the AI Apocolypse, especially from an employment perspective is real. Millions of people who lost their jobs during the Pandemic were replaced by AI and robots. Politicians from around the world, have touted plans to fight AI automation ...
Introduction
Hello world from your friendly neighborhood digital plumber! My name is Jason and welcome to my blog! Here you will find stories and thoughts surrounding the ever-evolving world of data engineering and machine learning. So a little bit about myself, ...