Advanced Data Platform Engineer

Design and implement complex data pipelines and distributed systems using Spark and Python. Apply software engineering best practices: clean code, modular design, CI/CD, automated testing, and code reviews. Develop and maintain lakehouse capabilities with Delta Lake and Iceberg, ensuring reliability and performance. Enable analytics workflows by integrating dbt for SQL transformations running on Spark. Collaborate with internal teams to deliver curated datasets and self-service analytics capabilities.

Optimize data warehousing solutions such as Databricks and Snowflake for performance and scalability. Implement observability and governance frameworks, including data lineage and compliance controls. Drive performance tuning, scalability strategies, and cost optimization across Spark jobs and cloud-native environments.