Job Description
Develop and maintain scalable data pipelines using Scala and Apache Spark, focusing on performance and reuse. Structure and evolve the medallion architecture (bronze, silver, gold), ensuring governance, traceability, and data quality. Implement solutions for data ingestion, transformation, and delivery in cloud and lakehouse environments. Define and apply data engineering best practices, including versioning, automated testing, and CI/CD. Support technical and architectural decisions with engineering and product teams. Participate in agile rituals and contribute to the continuous evolution of processes and tools. Monitor and optimize pipelines in production, ensuring reliability and efficiency. Work with tools like Airflow, Delta Lake, Iceberg, and the AWS environment. Investigate and propose improvements in architecture, performance, and data security.
About Serasa Experian
Serasa Experian is the first and largest Datatech in Brazil, leading in intelligence solutions for risk analysis and opportunities.