Data Engineer

Build infrastructure for ingestion, transformation, and loading an exponentially increasing volume of data from a variety of sources using Spark, SQL, AWS, and Databricks. The role includes building an organic entity resolution framework capable of correctly merging hundreds of billions of individual entities into a number of clean, consumable datasets, and developing CI/CD pipelines. Anomaly detection systems are utilized to continuously improve the quality of data we're pushing into production, as well as dreaming up solutions to largely undefined data engineering and data science problems.