Data Engineer, DaaS

PayPay's rapid growth necessitates the expansion of its product teams and underscores the critical need for a resilient Data Engineering Platform. This platform is vital to support increasing business demands. The Data Pipeline team is tasked with creating, deploying, and managing this platform, utilizing leading technologies like Databricks, Delta Lake, Spark, Scala, and the AWS suite. The Data Engineer will create and manage robust data ingestion pipelines leveraging Databricks, Airflow, Kafka, and Terraform. In addition, they will develop data processing workflows using Databricks, Snowflake, Delta Lake, and Spark technologies. The Data Engineer will also maintain and improve the Data Lakehouse, utilizing Unity Catalog for efficient data management and discovery, and collaborate across teams to facilitate smooth data flow and integration.