The DataDank 90-day plan
Master data engineering. Ace your interviews. Build your future.
A practical path to data engineering skills: clear tutorials, hands-on practice, interview questions, real-world projects, and structured roadmaps.
- Days
- 90
- Planned hours
- 348
- Topics tracked
- 1046
- Lessons
- 148
Your progress
Saved in this browser only, no account needed.Open a lesson or tick off an interview question and your progress appears here.
Continue where you left offThe roadmap
Three phases. Thirteen weeks. One goal.
Foundations first, then advanced topics and projects, then interview mastery. Each week starts with foundations, then adds tools after their prerequisites, with a weekly milestone.
Days 1–30 · 120 hours
Foundations
Build SQL, DSA, Python and core data-engineering foundations; finish one batch project and begin applications.
- Week 1Use SQL filtering, NULL handling and aggregation; solve 20 array/hash problems; explain core pipeline foundations; run the idempotent CSV loader locally.
- Week 2Solve 20 more DSA problems; join data safely, explain common file formats, define warehouse grain, and describe basic S3/IAM access.
- Week 3Build DataFrame transformations and explain lazy evaluation, SCD types and partitioning trade-offs.
- Week 4Explain the system-design method and sketch a batch ingestion framework after learning the storage foundations.
Target You can explain Idempotent CSV loader setup and replay check, complete today's SQL and DSA practice, and show the result of the hands-on exercise.
Open phase 1Days 31–60 · 114 hours
Core pipelines
Build reliable batch and CDC pipelines, practise cloud and warehouse work, and complete a second project.
- Week 5Implement incremental, replay-safe loads; explain CDC and Airflow backfills; start the warehouse pipeline project.
- Week 6Explain Delta Lake transactions and schema evolution, load data into a warehouse, and design a medallion lakehouse.
- Week 7Explain Kafka delivery basics, data contracts and quality checks; design an idempotent CDC pipeline.
- Week 8Tune Spark, describe CI/CD and observability, and complete two SQL/DSA or data-engineering mocks.
Target You can explain Retries & alerting, complete today's SQL and DSA practice, and show the result of the hands-on exercise.
Open phase 2Days 61–90 · 114 hours
Advanced and interview-ready
Practise streaming, system design and mock interviews; finish a project demo, résumé and portfolio.
- Week 9Explain streaming state and watermarks, complete a streaming project extension, and defend a clickstream design.
- Week 10Solve timed SQL/DSA sets and design a governed pipeline with quality and observability controls.
- Week 11Handle late data, retries and backfills in system-design interviews; explain operating costs and failure modes.
- Week 12Complete two mocks, explain project trade-offs clearly, and identify remaining weak areas.
- Week 13Finish the project demo, résumé and portfolio; complete final mocks and a focused revision plan.
Target You can explain Structured Streaming model, complete today's SQL and DSA practice, and show the result of the hands-on exercise.
Open phase 3
Courses
Pick a technology and start learning
Each course has ordered lessons, interview questions, projects and a cheat sheet.
- Languages & querySQLSQL is the core language of data work: querying, transforming and modelling data in warehouses, lakehouses and Spark. Start here before any other tool.
- Languages & queryPythonPython glues pipelines together: ingestion, validation, orchestration and PySpark jobs. Focus on functions, generators, error handling and testable code.
- Distributed processingPySparkPySpark is the Python API for Apache Spark. Learn DataFrames, joins, window functions and how partitions and shuffles decide performance.
- Distributed processingApache SparkHow Spark works inside: driver and executors, RDDs, jobs and stages, shuffles and skew, caching, Catalyst, AQE, memory tuning, Kubernetes, History Server.
- Streaming & orchestrationKafkaKafka is a distributed log used for streaming data. Learn topics, partitions, consumer groups and delivery semantics before building streaming pipelines.
- Streaming & orchestrationAirflowAirflow schedules and orchestrates pipelines as DAGs. Learn scheduling, task dependencies, retries and idempotent task design.
- Data platformsDelta LakeDelta Lake adds ACID transactions, schema enforcement and time travel to files in a data lake, which is the foundation of the lakehouse pattern.
- Data platformsData modelingData modeling and warehousing: star schemas and grain, fact and dimension design, SCDs, Data Vault, dbt, semantic layers and incremental models.
- Data platformsETL and ELTETL transforms data before loading it; ELT loads first and transforms inside the warehouse or lakehouse. Learn when each fits.
- Data platformsDatabricksDatabricks is a lakehouse platform built around Apache Spark and Delta Lake. Learn its workspace, compute, jobs and Unity Catalog governance model.
- Data platformsSnowflakeSnowflake is a cloud data warehouse that separates storage from compute. Learn virtual warehouses, micro-partitions, pruning, caching and cost control.
- CloudAWSAWS for Data Engineers: S3, IAM, Lambda, Glue, Athena, Redshift, EMR, Kinesis, Step Functions, Lake Formation and how they fit into one data platform.
- Interview foundationsDSADSA for Data Engineering interviews: easy-to-medium problems on arrays, hashing, strings, two pointers and sliding windows, sometimes graphs or DP.
Roadmap
Data Engineer roadmap
From beginner foundations to interview-ready, in 13 ordered stages.
Interview
Three focused ways to prepare
- Technology interviewsInterview questionsDirect answers first, then reasoning, common mistakes and follow-ups.
- System designCase studiesRequirements, architecture, reliability and trade-offs for real data platforms.
- Company preparationCompany guidesPreparation that separates attributed information from practice questions.
Projects
Build something you can explain
- BeginnerCSV to Data Warehouse PipelineA small online shop exports orders as daily CSV files. Build a pipeline that loads them into a star schema so that sales can be reported reliably, even when files are resent, corrected or contain bad rows.
- IntermediateE-commerce Analytics Data PlatformAn online store has orders, customers, products, refunds and web sessions in separate systems. Build an ELT platform that models them into trusted marts for revenue, conversion and repeat customers, with every metric defined once and tested.
- AdvancedChange Data Capture PipelineReplicate an operational PostgreSQL table into a lakehouse table within minutes, including updates and deletes, so analysts query current data without touching the production database, and prove that replays, duplicates and out-of-order delivery cannot corrupt the copy.
Featured guides
Start with these
- Career guideBeginnerHow to Explain a Data Engineering Project in an InterviewA repeatable structure for explaining a data engineering project in an interview: problem, design, trade-offs, failure handling and honest results.
- ArticleBeginnerStar, Snowflake and Galaxy Schemas and the GrainBuild a star schema for an online shop, declare the grain, snowflake a dimension, and query a galaxy of fact tables without double counting. Verified SQL.
- ArticleBeginnerKafka Topics, Partitions and Consumer GroupsHow Kafka topics split into partitions, how keys decide ordering, what replication factor protects, and how consumer groups divide partitions between consumers.
- ArticleIntermediatePySpark Window Functions: Ranking, Lag and Running TotalsPySpark window functions for top-N per group, deduplication, lag and lead, running totals, gap filling and sessions, and avoiding single-partition windows.
How we write
Editorial principles
Reviewed technical content
Claims about tools are checked against official documentation, and each page shows when it was last reviewed.
Version-aware
Where behaviour depends on a version or platform release, the page says which one it describes.
Evidence-labelled interviews
Company material is labelled as attributed, reported or representative practice. We never present invented questions as real ones.
Structured paths
Every lesson sits in a course with a clear previous and next step.