Setup AWS MWAA in 7 Easy Steps
How to set up Airflow in a few mins vs hours or days.
Apr 12, 20225 min read
Command Palette
Search for a command to run...
Latest articles
Alley-oop
DE tosses up the data to DS for the Business Value Alley-oop!
Dec 9, 20211 min readApache Airflow, Not Just for Data Pipelines
Alternate Use Cases
Sep 1, 20213 min readData Versioning Your Processed Data
Does your team or company track the versions of your processed data? If not, maybe one should consider the added value of data versioning. When I use the term "data versioning", what I mean is tracking how your data such as an entity has changed ove...
Jul 27, 20214 min readBrickit: Computer Vision App that Shows You Lego Ideas.
Like many of us "nerds" out there, we probably have a bucket or three of Legos sitting somewhere. Most of my collectable "Adult" Legos get built once based on the set instructions. However, if you're looking to build something new and interesting wit...
Jul 1, 20211 min readIncrease Data Processing Throughput of a Step Function with Dynamic Parallelism
Is your data growing beyond what a Lambda-based Step Function Lambda can handle? Are you having problems with AWS Step Functions timing out in 900 seconds because your Lambda function was too slow at processing data? Maybe your data is large but not ...
Jun 29, 20212 min readHow to Not Lose Your Job to AI: Don't Be An Endpoint
The fear of the AI Apocolypse, especially from an employment perspective is real. Millions of people who lost their jobs during the Pandemic were replaced by AI and robots. Politicians from around the world, have touted plans to fight AI automation ...
Jun 22, 20213 min readIntroduction
Hello world from your friendly neighborhood digital plumber! My name is Jason and welcome to my blog! Here you will find stories and thoughts surrounding the ever-evolving world of data engineering and machine learning. So a little bit about myself, ...
Jun 8, 20211 min read