Data Engineer

We are looking for a Mid Level/Sr Data Engineer located in Brazil to work on a financial project for one of our largest clients. You will be responsible for building and maintaining data ingestion pipelines in a cloud environment, using Azure Databricks. The professional will be responsible for integrating data from various sources, ensuring quality, governance, and performance. Responsibilities include developing and maintaining data ingestion pipelines in Azure Databricks, integrating data from diverse sources (APIs, relational databases, files, etc.), working with cloud-based architectures (preferably Azure), collaborating with business and analytics teams to understand data requirements, ensuring data quality, security, and governance, and documenting processes and best practices. Technical requirements include experience with Azure Databricks, Azure Data Lake, and Spark, proficiency in Python for pipeline development, experience with data modeling, ingestion, and transformation, knowledge of data architecture in the cloud (preferably Azure), experience with code versioning (Git) and DevOps practices (CI/CD), and familiarity with pipeline monitoring and logging strategies.