Data Engineer

Be crucial in accelerating our efforts to build standalone data products that enable data teams and independent developers to create innovative solutions at massive scale. In this role, you will be working with a team to continuously improve our existing datasets as well as pursuing new ones. We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things.

Use and develop web crawling technologies to capture and catalog data on the internet. Support and improve our web crawling infrastructure. Structure, define, and model captured data, providing semantic data definition and automate data quality monitoring for data that we crawl. Develop new techniques to increase speed, efficiency, scalability, and reliability of web crawls. Use big data processing platform to build data pipelines, publish data, and ensure the reliable availability of data that we crawl. Work with our data product and engineering team to design and implement new data products with captured data, and enhance and improve upon existing products.