Source Job

$217,000–$303,900/yr
US

  • Lead the development of Reddit's Ingestion Platform, designing and delivering reliable software for distributed data movement across streaming and batch workloads.
  • Own the architecture of the platform's control and data planes, including pipeline APIs, connectors, and sink integrations, expanding beyond Kafka-to-BigQuery to S3/GCS and Apache Iceberg.
  • Mentor engineers, drive migrations from legacy systems, and establish robust reliability, security, and operational practices for pipelines running on Kubernetes.

Go Python Java Kafka Kubernetes

20 jobs similar to Staff Software Engineer - Ingestion Platform

Jobs ranked by similarity.

$164,200–$229,900/yr
US

  • Refine and maintain data infrastructure for ML and analytics workflows on data from hundreds of millions of users.
  • Own the Data Movement Platform enabling batch and stream processing, investing in Spark, Flink, and Airflow technologies.
  • Build automated solutions to minimize toilsome work, providing a declarative, self-service experience for data users.

Reddit is a community of communities built on shared interests, passion, and trust, home to the most open and authentic conversations on the internet. With over 100,000 active communities and approximately 130 million daily active unique visitors, Reddit is one of the internet's largest sources of information, fostering a flexible and inclusive culture.

Romania 5w PTO 16w maternity 6w paternity

  • Design and build distributed data systems handling large-scale ingestion and processing.
  • Drive architectural decisions and take end-to-end ownership of critical components.
  • Collaborate with product teams to translate ambiguous requirements into robust technical solutions.

Our partner builds a large-scale, multi-chain data platform that ingests, models, and delivers blockchain data to users and developers. They are a remote-first, distributed team with a strong engineering culture focused on ownership and collaboration.

United States

  • Partner with cross-functional teams to design and deliver scalable backend systems for major product initiatives.
  • Own the full software lifecycle from technical design to rollout, using A/B experiments and data analysis to drive decisions.
  • Build and maintain high-performance APIs and distributed services using modern languages and tools.

Reddit is a community of communities, built on shared interests, passion, and trust. It is home to the most open and authentic conversations on the internet, with 100,000+ active communities and approximately 130 million daily active unique visitors.

Global

  • Design and implement core components of the streaming platform including Kafka-compatible APIs and storage systems.
  • Solve complex distributed systems problems involving performance optimization and reliability.
  • Debug production issues, collaborate with customers, and contribute to open-source projects.

StreamNative, founded by the creators of Apache Pulsar, is redefining real-time data streaming with its URSA engine. The company is a growing startup focused on building a cloud-native streaming platform with a culture of engineering excellence.

Americas

  • Design, build, and evolve the core data platform infrastructure including distributed query engines and orchestration.
  • Own our lakehouse infrastructure as code using Terraform and Ansible on Kubernetes.
  • Build and maintain low-latency streaming and batch ingestion pipelines, and scale the BI landscape.

Alpaca is a US-headquartered global leader in agent-first brokerage infrastructure for stocks, ETFs, options, crypto, and more. We are a dynamic team of 400+ globally distributed members who thrive working from our favorite places around the world.

Global Unlimited PTO 16w maternity 16w paternity

  • Lead migration of Vanta's resource data model from Mongo to Postgres while preserving APIs and customer evidence.
  • Design and evolve data ingestion pipeline for terabyte-scale streams with distributed systems.
  • Mentor senior engineers and set architectural direction for streaming infrastructure, caching, and storage.

Vanta helps businesses earn and prove trust through automated security monitoring and compliance. We have a kind and talented team, founded in 2018 to restore trust in internet businesses.

India

  • Own and drive impactful distributed systems problems end-to-end, from inception through production launch.
  • Collaborate with a strong engineering team to prioritize and solve the most important problems for the company.
  • Raise the quality bar while keeping systems reliable and operationally lean, and mentor fellow engineers.

StarTree is a cloud-based software company that enables businesses to derive advanced insights from real-time and historical data using Apache Pinot. The company was founded by the core engineering team behind Apache Pinot, has secured Series B funding, and was named one of The Information's 50 Most Promising Startups.

$212,000–$265,000/yr
US

  • Design and build the next generation big data compute platform for ETL, analytics, and machine learning at Airbnb.
  • Operate, manage, and improve the reliability, performance, observability, and cost efficiency of the data platform.
  • Write maintainable, self-documenting code, perform code reviews, and contribute to open source software.

Airbnb is a global home-sharing platform that connects travelers with unique accommodations and experiences. The company has grown to over 5 million hosts and 2 billion guest arrivals, fostering a culture of belonging and innovation.

$139,200–$235,200/yr
Canada United States Unlimited PTO

  • Design, build, and operate GitLab Orbit backend services, primarily in Rust, within a distributed, cloud-native environment.
  • Improve deployment, monitoring, and operations using Kubernetes, Helm, Terraform, and cloud services from AWS or GCP.
  • Automate operational work, strengthen observability, and manage production issues to reduce single points of failure.

GitLab is the intelligent orchestration platform for DevSecOps, helping organizations increase developer productivity, improve operational efficiency, and accelerate digital transformation. Trusted by more than 50 million registered users and over 50% of the Fortune 100, GitLab fosters a high-performance culture driven by shared values and continuous knowledge exchange.

$217,000–$303,900/yr
US 17w maternity 17w paternity

  • Lead reliability initiatives across multiple Ads domains including ad serving, auctions, targeting, reporting, measurement, and billing.
  • Design and build platforms, tooling, and automation that improve reliability and developer productivity at scale.
  • Participate in on-call rotations, lead complex incident investigations and coordinate cross-functional response efforts during major production events.

Reddit is a community of communities, built on shared interests, passion, and trust. With 100,000+ active communities and approximately 130 million daily active unique visitors, Reddit is one of the internet's largest sources of information.

$77,000–$106,700/yr
France

  • Design and develop scalable search and indexing systems for an AI search engine.
  • Ensure operational excellence by participating in on-call rotation and maintaining system quality.
  • Collaborate with a global remote team to solve distributed system challenges.

Algolia is a pioneer and market leader in AI Search, empowering over 18,000 businesses to deliver blazing-fast search experiences. With $150 million in Series D funding and a valuation of $2.25 billion, the company fosters a high-trust, flexible culture and values diversity and collaboration.

Poland 5w PTO 20w maternity 16w paternity

  • Lead backend engineering for the platform team using Java, Python, and distributed systems.
  • Drive technical roadmaps and improve processes, systems, and tools to deliver quality products.
  • Mentor engineers across the team and foster a positive, collaborative culture.

Sprout Social is a global leader in social media management and analytics software. Founded in 2010, the company has a hybrid team of 1,400 employees across the globe and is consistently recognized as a best place to work.

$95,460–$139,860/yr
Canada

  • Design, build, and improve systems to reliably and efficiently process data at the scale of tens of terabytes per day.
  • Evaluate new technologies and set best-practice standards with Data Engineering and Data Platform Engineering.
  • Drive cross-functional projects with stakeholders across Mozilla and mentor teammates to develop engineering best practices.

Mozilla Corporation is a non-profit-backed technology company that makes pioneering brands like Firefox, the privacy-minded web browser, and works on diverse areas including AI, social media, security. The company is wholly owned by the non-profit Mozilla Foundation, with over 225 million monthly users, and focuses on making the internet better for people.

$151,000–$206,000/yr
US Canada Unlimited PTO

  • Build large-scale real-time services and applications leveraging massive datasets.
  • Develop and maintain data pipelines, messaging systems, databases, and cloud services.
  • Work with Machine Learning Engineers and Security Researchers on security solutions.

Censys provides real-time Internet intelligence and threat insights to global governments and Fortune 500 companies. It is a growing company with a focus on comprehensive internet mapping and security solutions.

Global Unlimited PTO

  • Design and build high-throughput data replication integrations with databases, data lakes, and warehouses.
  • Own end-to-end reliability for production systems, including debugging infrastructure and on-call.
  • Collaborate with customers and cross-functional teams to drive product innovation and roadmap.

ClickHouse develops an open-source column-oriented database management system and offers the ClickPipes platform for real-time data replication. The company is globally distributed across over 25 countries with a hyper-growth startup culture that values open source, continuous learning, and in-person connection.

$217,000–$303,000/yr
US

  • Lead the team that architects, builds, and operates the Notifications Platform at Reddit.
  • Set technical direction for shared platform capabilities spanning notification APIs, eventing, and delivery pipelines.
  • Collaborate with product and engineering teams to define platform contracts and improve developer experience.

Reddit is a community of communities where users submit, vote, and comment on topics they care about. With 100,000+ active communities and approximately 130 million daily active unique visitors, it is one of the internet's largest sources of information.

India

  • Lead high-impact projects with real business outcomes, owning ERP data management and building connectors to ensure high availability and secure storage.
  • Manage large-scale email processing systems handling tens of millions of accounts and hundreds of millions of emails daily, with expertise in IMAP, SMTP, Redis, and MySQL.
  • Bridge business needs and technical implementation, designing backend APIs and maintaining distributed systems with a focus on reliability and scalability.

YipitData is the leading market research and analytics firm for the disruptive economy, recently raising $475M from The Carlyle Group at a valuation over $1B. Our award-winning, people-centric culture emphasizes transparency, ownership, and continuous mastery, recognized by Inc. as a Best Workplace for three consecutive years.

US Unlimited PTO

  • Design, build, and operate the ZMP real-time events pipeline for high-volume event data.
  • Provide technical leadership, own architecture, and mentor teammates through code and design reviews.
  • Collaborate with product, data, and consumer teams to turn SLAs into working systems.

Zeta Global is an AI-powered marketing cloud that leverages artificial intelligence and trillions of consumer signals to help marketers acquire, grow, and retain customers more efficiently. Founded in 2007 and headquartered in New York City with offices worldwide, the company fosters a culture of trust and belonging with a focus on diversity and inclusion.

US Unlimited PTO

  • Build and deliver high-quality solutions that power Honeycomb's query and data storage infrastructure.
  • Scope and deliver projects independently, breaking down complex storage problems into achievable steps.
  • Support our services in production, participating in on-call rotations and reducing toil.

Honeycomb is a service defining observability and raising expectations for developer tools, working with companies like HelloFresh and Slack. We've scaled past 200 people, closed Series D funding, and were named to Forbes' America's Best Startups in 2022 and 2023.

$180,000–$240,000/yr
US

  • Own large slices of the system end to end, from approach to operation.
  • Turn Beads into a platform and take Gas City to the cloud.
  • Define SLOs, observability, backups, and security baseline for enterprise readiness.

Gas City builds the open-source stack teams use to run coding agents at scale, including the Beads work graph and Gas City agent orchestration. It's a small, flat organization moving toward revenue with a focus on reliability and agent-driven development.