Job Description
The Lead Data Engineer will design and develop scalable data pipelines using PySpark to support analytics and reporting needs. They will write efficient SQL and Python code to transform, cleanse, and optimize large datasets. The role involves collaboration with machine learning engineers, product managers, and developers to understand data requirements and deliver solutions. The engineer will implement and maintain robust ETL processes to integrate structured and semi-structured data from various sources, ensuring data quality, integrity, and reliability across pipelines and systems. They will also participate in code reviews, troubleshooting, and performance tuning, working independently to identify and resolve data-related issues. Contribution to Azure-based data solutions and support for cloud migration initiatives and DevOps practices are also expected. Guidance on best practices and mentoring junior team members may be required.
About Bertoni
We are a multinational team of individuals who believe that, with the right knowledge and approach, technology is the answer to the challenges businesses face today.