The DataDank 90-day plan
Master data engineering. Ace your interviews. Build your future.
A practical path to data engineering skills: clear tutorials, hands-on practice, interview questions, real-world projects, and structured roadmaps.
- Days
- 90
- Planned hours
- 270
- Topics tracked
- 1046
- Lessons
- 175
Your progress
Saved in this browser only, no account needed.Open a lesson or tick off an interview question and your progress appears here.
Continue where you left offThe roadmap
Three phases. Thirteen weeks. One goal.
Foundations first, then advanced topics and projects, then interview mastery. Each week starts with foundations, then adds tools after their prerequisites, with a weekly milestone.
Days 1–28 · 84 hours
Foundations
SQL, Python and easy DSA; data-engineering concepts and data modeling; start Spark; build a tested CSV-to-warehouse pipeline.
- Week 1Write filtered and aggregated SQL, use Python for files and functions, and solve easy array and hashing problems.
- Week 2Join tables safely, structure Python with tests and logging, and explain file formats and batch vs streaming.
- Week 3Design a star schema at the right grain, handle slowly changing dimensions, and load data idempotently.
- Week 4Write window-function SQL, explain Spark's execution model, and finish the CSV-to-warehouse project.
Target You can explain Python for data engineering, and you solved today's SQL and DSA problems without hints.
Open phase 1Days 29–63 · 105 hours
Core pipelines
Spark, Delta Lake, Airflow, dbt and Snowflake; medium DSA; batch and analytics projects; begin system design.
- Week 5Transform data with PySpark DataFrames, choose join strategies, and start system design with batch ingestion.
- Week 6Diagnose shuffles and skew in Spark, explain Delta Lake transactions, and finish the large-scale batch project.
- Week 7Write Airflow DAGs with safe retries and backfills, build dbt models, and design an ELT pipeline.
- Week 8Schedule with sensors and assets in Airflow, test dbt models, and build the e-commerce analytics project.
- Week 9Load and model data in Snowflake, use streams and tasks, and design analytics platforms.
Target You can explain Reading/writing Parquet, and you solved today's SQL and DSA problems without hints.
Open phase 2Days 64–90 · 81 hours
Streaming, design and interviews
Kafka, Spark streaming, AWS services and system design; graph and DP problems; a streaming project, mocks and final revision.
- Week 10Explain Kafka partitions, consumer groups and delivery guarantees; build the S3, PySpark and Snowflake pipeline.
- Week 11Explain Kafka replication and exactly-once, use AWS ingestion services, and design streaming systems.
- Week 12Build a watermarked Spark streaming job, design governed platforms, and run the streaming project end to end.
- Week 13Revise every track, solve hard DSA problems, and finish your project demo, résumé and mocks.
Target You can explain Clustering keys, and you solved today's SQL and DSA problems without hints.
Open phase 3
Courses
Pick a technology and start learning
Each course has ordered lessons, interview questions, projects and a cheat sheet.
- Languages & querySQLSQL is the core language of data work: querying, transforming and modelling data in warehouses, lakehouses and Spark. Start here before any other tool.
- Languages & queryPythonPython glues pipelines together: ingestion, validation, orchestration and PySpark jobs. Focus on functions, generators, error handling and testable code.
- Distributed processingPySparkPySpark is the Python API for Apache Spark. Learn DataFrames, joins, window functions and how partitions and shuffles decide performance.
- Distributed processingApache SparkHow Spark works inside: driver and executors, RDDs, jobs and stages, shuffles and skew, caching, Catalyst, AQE, memory tuning, Kubernetes, History Server.
- Streaming & orchestrationApache KafkaApache Kafka is a distributed log used for streaming data. Learn topics, partitions, consumer groups and delivery semantics before building streaming pipelines.
- Streaming & orchestrationApache AirflowApache Airflow schedules and orchestrates pipelines as DAGs. Learn scheduling, task dependencies, retries and idempotent task design.
- Data platformsDelta LakeDelta Lake adds ACID transactions, schema enforcement and time travel to files in a data lake, which is the foundation of the lakehouse pattern.
- Data platformsData modelingData modeling and warehousing: star schemas and grain, fact and dimension design, SCDs, Data Vault, dbt, semantic layers and incremental models.
- Data platformsETL and ELTETL transforms data before loading it; ELT loads first and transforms inside the warehouse or lakehouse. Learn when each fits.
- Data platformsDatabricksDatabricks is a lakehouse platform built around Apache Spark and Delta Lake. Learn its workspace, compute, jobs and Unity Catalog governance model.
- Data platformsSnowflakeSnowflake is a cloud data warehouse that separates storage from compute. Learn virtual warehouses, micro-partitions, pruning, caching and cost control.
- CloudAWSAWS for Data Engineers: S3, IAM, Lambda, Glue, Athena, Redshift, EMR, Kinesis, Step Functions, Lake Formation and how they fit into one data platform.
- Interview foundationsDSADSA for Data Engineering interviews: easy-to-medium problems on arrays, hashing, strings, two pointers and sliding windows, sometimes graphs or DP.
- Distributed processingdbtdbt turns SQL select statements into tested, documented warehouse models. Learn models, materializations, incremental models, tests, snapshots and CI.
- Interview foundationsDev toolsThe command line, shell scripts, Git, Python environments, Docker and repo automation: the day-one toolkit every data pipeline job assumes you already know.
Roadmap
Data Engineer roadmap
From beginner foundations to interview-ready, in 13 ordered stages.
Interview
Three focused ways to prepare
- Technology interviewsInterview questionsDirect answers first, then reasoning, common mistakes and follow-ups.
- System designCase studiesRequirements, architecture, reliability and trade-offs for real data platforms.
- Company preparationCompany guidesPreparation that separates attributed information from practice questions.
Projects
Build something you can explain
- BeginnerCSV to Data Warehouse PipelineA small online shop exports orders as daily CSV files. Build a pipeline that loads them into a star schema so that sales can be reported reliably, even when files are resent, corrected or contain bad rows.
- IntermediateE-commerce Analytics Data PlatformAn online store has orders, customers, products, refunds and web sessions in separate systems. Build an ELT platform that models them into trusted marts for revenue, conversion and repeat customers, with every metric defined once and tested.
- AdvancedChange Data Capture PipelineReplicate an operational PostgreSQL table into a lakehouse table within minutes, including updates and deletes, so analysts query current data without touching the production database, and prove that replays, duplicates and out-of-order delivery cannot corrupt the copy.
Featured guides
Start with these
- Career guideBeginnerHow to Explain a Data Engineering Project in an InterviewA repeatable structure for explaining a data engineering project in an interview: problem, design, trade-offs, failure handling and honest results.
- ArticleBeginnerStar, Snowflake and Galaxy Schemas and the GrainBuild a star schema for an online shop, declare the grain, snowflake a dimension, and query a galaxy of fact tables without double counting. Verified SQL.
- ArticleBeginnerKafka Topics, Partitions and Consumer GroupsHow Kafka topics split into partitions, how keys decide ordering, what replication factor protects, and how consumer groups divide partitions between consumers.
- ArticleIntermediatePySpark Window Functions: Ranking, Lag and Running TotalsPySpark window functions for top-N per group, deduplication, lag and lead, running totals, gap filling and sessions, and avoiding single-partition windows.
How we write
Editorial principles
Reviewed technical content
Claims about tools are checked against official documentation, and each page shows when it was last reviewed.
Version-aware
Where behaviour depends on a version or platform release, the page says which one it describes.
Evidence-labelled interviews
Company material is labelled as attributed, reported or representative practice. We never present invented questions as real ones.
Structured paths
Every lesson sits in a course with a clear previous and next step.