Digital Plumbing by Jason

Digital Plumbing by Jason

Follow
homebadges

Setup AWS MWAA in 7 Easy Steps

Apr 12, 20225 min read

How to set up Airflow in a few mins vs hours or days. · If you have ever used Airflow, you know that it should exist in a cluster environment. Sure you...

Setup AWS MWAA in 7 Easy Steps

Alley-oop

Dec 9, 20211 min read

DE tosses up the data to DS for the Business Value Alley-oop!

Alley-oop

Apache Airflow, Not Just for Data Pipelines

Sep 1, 20213 min read

Alternate Use Cases · If you come from the world of data engineering or if your company ingests and processes large quantities of data on a regular...

Apache Airflow, Not Just for Data Pipelines

Data Versioning Your Processed Data

Jul 27, 20214 min read

Does your team or company track the versions of your processed data? If not, maybe one should consider the added value of data versioning. When I use...

Data Versioning Your Processed Data

Brickit: Computer Vision App that Shows You Lego Ideas.

Jul 1, 20211 min read

Like many of us "nerds" out there, we probably have a bucket or three of Legos sitting somewhere. Most of my collectable "Adult" Legos get built once...

Brickit: Computer Vision App that Shows You Lego Ideas.

Increase Data Processing Throughput of a Step Function with Dynamic Parallelism

Jun 29, 20212 min read

Is your data growing beyond what a Lambda-based Step Function Lambda can handle? Are you having problems with AWS Step Functions timing out in 900...

Increase Data Processing Throughput of a Step Function with Dynamic Parallelism