Data Engineer III

Architecture and Design:

• Design, implement, and evolve onX’s Iceberg-based lakehouse architecture to balance scalability, cost, and performance.
• Define and implement metadata first and semantic layer architectures that make data understandable, trusted, and ready for self-service analytics.

Data Pipeline Development:

• Build and maintain scalable, reliable ingestion and transformation pipelines using GCP tools (Spark, Dataflow, Pub/Sub, BigQuery, Dataplex, Cloud Composer).
• Develop batch and streaming frameworks with schema enforcement, partitioning, and lineage capture.
• Apply data quality checks and contracts at every layer to ensure consistency, auditability, and trust.

MLOps and Advanced Workflows:

• Collaborate with Data Science to integrate feature stores, model registries, and model monitoring into the platform.
• Build and maintain standardized orchestration and observability patterns for both data and ML pipelines, ensuring SLA, latency, and cost visibility.
• Develop reusable microservices that support model training, deployment, and scoring within a governed, observable MLOps framework.