I consider these tools the New Data Engineering the current standard :
- Scheduling Tools: Airflow
- ETL-adjacent processes: dbt
- Data Quality Testing: Great Expectations
- Infrastructure: Terraform
- Data Catalog/Discovery: Amundsen
Here’s a visual guideline for modern data engineer roadmap
https://github.com/datastacktv/data-engineer-roadmap
credit to reddit /r/dataengineering
