Data Engineer – Databricks

Design, build, and maintain scalable data pipelines using Databricks and Apache Spark. Integrate data from various sources into data lakes or data warehouses. Implement and manage Delta Lake architecture for reliable, versioned data storage. Ensure data quality, performance, and reliability through testing and monitoring. Collaborate with data analysts, scientists, and stakeholders to meet data needs. Automate workflows and manage job scheduling within Databricks. Maintain clear and thorough documentation of data workflows and architecture. Work on Databricks-based AI/ML solutions , including machine learning pipelines , in collaboration with data science teams.