Category Archives: Uncategorized

The Data Engineering 2021

Posted on April 13, 2021 by mx

I consider these tools the New Data Engineering the current standard : Scheduling Tools: Airflow ETL-adjacent processes: dbt Data Quality Testing: Great Expectations Infrastructure: Terraform Data Catalog/Discovery: Amundsen Here’s a visual guideline for modern data engineer roadmap https://github.com/datastacktv/data-engineer-roadmap credit to … Continue reading →

Posted in Uncategorized | Leave a comment

How Netflix does data in AWS

Posted on July 17, 2020 by mx

https://netflixtechblog.com/byte-down-making-netflixs-data-infrastructure-cost-effective-fee7b3235032

Posted in Uncategorized | Leave a comment

GCP for AWS professionals

Posted on May 5, 2020 by mx

Someone made a GCP lookup list for AWS cloud people Service comparisons The following table provides a side-by-side comparison of the various services available on AWS and Google Cloud. Service Category Service AWS Google Cloud Compute IaaS Amazon Elastic Compute … Continue reading →

Posted in Uncategorized | Leave a comment

Modern Data Engineering is Complicated

Posted on April 27, 2020 by mx

Modern Data Engineering is Complicated. There are so many things to know to be good. Languages : SQL , Python, Scala Operating Systems : Linux, bash shell Cloud : AWS, Azure, GCP Data Pipelines : Airflow, Kubeflow DevOps : Kubernetes, … Continue reading →

Posted in Uncategorized | Leave a comment

My other hobby is Stocks

Posted on February 24, 2020 by mx

My other hobby is investing and picking stocks.Here’s my other blog: http://hunandelightmd.com/

Posted in Uncategorized | Leave a comment

mysql tricks: do instant table swap to mitigate mysql deadlock error.

Posted on May 19, 2017 by mx

mysql tricks: do instant table swap to mitigate mysql deadlock error. Continue reading →

Posted in etl, relational databases, Uncategorized | Tagged , , , , | Leave a comment

Slide Share’s full of useful knowledge

Posted on February 20, 2017 by mx

Best practices for PySpark programming Programming in Spark using PySpark from Mostafa Elzoghbi History of SQL and all the advanced features over the last 30 years among big vendors. Modern SQL in Open Source and Commercial Databases from Markus Winand

Posted in spark, Uncategorized | Tagged , , , | Leave a comment

The reality of a data worker.

Posted on January 23, 2017 by mx

Taken from a Dataiku meetup slide.  This picture hit close to home.

Posted in data wrangling, Uncategorized, wheresthejoke | Tagged | Leave a comment

Things to note when migrating web hosts

Posted on January 23, 2017 by mx

Things to note when migrating web hosts Continue reading →

Posted in Uncategorized | Leave a comment

New Year, New Site

Posted on January 16, 2017 by mx

I switched to wordpress.com as my host. I will most likely switch to AWS later.

Posted in Uncategorized | Leave a comment