Source Job

US

  • Design, build, and maintain scalable ETL pipelines for large-scale data processing.
  • Implement data transformations and workflows using PySpark at an intermediate to advanced level.
  • Optimize pipelines for performance, scalability, and cost efficiency across environments.

PySpark Databricks ETL Data Pipelines

20 jobs similar to Data Engineer

Jobs ranked by similarity.

US

  • Design, build, and optimize ETL/ELT workflows using Databricks, SQL, and Python/PySpark.
  • Develop and maintain robust, scalable, and efficient data pipelines for processing large datasets.
  • Collaborate with cross-functional teams to deliver impactful data solutions.

Jobgether is an AI-powered platform that helps job seekers find suitable opportunities. They connect top-fitting candidates with hiring companies, streamlining the recruitment process through objective and fair assessments.

US Unlimited PTO

  • Partner with clients and implementation teams to understand data distribution requirements.
  • Design and develop data pipelines integrating with Databricks and Snowflake, ensuring accuracy and integrity.
  • Lead architecture and implementation of solutions for health plan clients, optimizing cloud-based technologies.

Abacus Insights is changing the way healthcare works by unlocking the power of data to enable the right care at the right time. Backed by $100M from top VCs, they're tackling big challenges in an industry that’s ready for change with a bold, curious, and collaborative team.

South America

  • Design, develop, and maintain scalable and robust data pipelines.
  • Create solutions for data ingestion, transformation, and modeling using Databricks, Spark/PySpark, Cloudera, and Azure Data Factory (ADF).
  • Ensure the quality, integrity, and usability of data throughout the entire pipeline.

CI&T specializes in technological transformation, uniting human expertise with AI to create scalable tech solutions. With over 8,000 CI&Ters worldwide, they have partnered with over 1,000 clients during their 30-year history, with a focus on Artificial Intelligence.

$120,000–$160,000/yr
US

  • Design and implement scalable, reliable, and efficient data pipelines to support clinical, operational, and business needs.
  • Optimize data storage and processing in data lakes and cloud data warehouses (Azure, Databricks).
  • Proactively suggest improvements to infrastructure, processes, and automation to improve system efficiency, reduce costs, and enhance performance.

Care Access is dedicated to ensuring that every person has the opportunity to understand their health, access the care they need, and contribute to the medical breakthroughs of tomorrow. They are working to make the future of health better for all and have hundreds of research locations, mobile clinics, and clinicians across the globe.

Europe

  • Design, build, and maintain scalable, high-quality data pipelines.
  • Implement robust data ingestion, transformation, and storage using cloud-based technologies.
  • Collaborate with stakeholders to understand business goals and translate them into data engineering solutions.

CI&T is a tech transformation specialist, uniting human expertise with AI to create scalable tech solutions. With over 8,000 employees around the world, they have partnerships with more than 1,000 clients and value diversity, fostering a diverse, inclusive, and safe work environment.

$190,800–$267,100/yr
US

  • Be the Analytics Engineering lead within the Sales and Marketing organization.
  • Be the data steward for Sales and Marketing: architect and improve the data collection.
  • Develop and maintain robust data pipelines and workflows for data ingestion and transformation.

Reddit is a community-driven platform built on shared interests and trust, fostering open and authentic conversations. With over 100,000 active communities and approximately 116 million daily active unique visitors, it serves as a major source of information on the internet.

$135,000–$165,000/yr
US Unlimited PTO

  • Design, build, and maintain scalable data pipelines.
  • Develop and implement data models for analytical use cases.
  • Implement data quality checks and governance practices.

MO helps government leaders shape the future. They engineer scalable, human-centered solutions that help agencies deliver their mission faster and better. They are building a company where technologists, designers, and builders can serve the mission and grow their craft.

$110,572–$145,000/yr
US Unlimited PTO

  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design data models and schemas that facilitate data analysis and reporting
  • Design, develop, and maintain scalable and efficient data pipelines and ETL processes to ingest, process, and transform large volumes of data from various sources into usable formats
  • Build and optimize data storage and processing systems, including data warehouses, data lakes, and big data platforms, using AWS services such as Amazon Redshift, AWS Glue, AWS EMR, AWS S3, and AWS Lambda, to enable efficient data retrieval and analysis

ATPCO is the world's primary source for air fare content. They hold over 200 million fares across 160 countries and the travel industry relies on their technology and data solutions. ATPCO believes in flexibility, trust, and a culture where your wellbeing comes first.

$90,200–$130,800/yr
US 3w PTO

  • Lead support of client’s Azure Data platform and Power BI Environment.
  • Consult, develop, and advise on solutions in Microsoft Azure and Power BI.
  • Proactively mentors junior team members and actively gives feedback.

3Cloud is a company where people roll up their sleeves to take on tough problems together. They hire people who aren’t afraid to experiment or fail and who are willing to give direct and candid feedback, so they can deliver amazing experiences and solutions to their clients.

$115,000–$145,000/yr
US

  • Collaborate with business leaders, engineers, and product managers to understand data needs.
  • Design, build, and scale data pipelines across a variety of source systems and streams (internal, third-party, as well as cloud-based), distributed/elastic environments, and downstream applications and/or self-service solutions
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

NBCUniversal is one of the world's leading media and entertainment companies that creates world-class content, which we distribute across our portfolio of film, television, and streaming, and bring to life through our global theme park destinations, consumer products, and experiences. We champion an inclusive culture and strive to attract and develop a talented workforce to create and deliver a wide range of content reflecting our world.

US Unlimited PTO

  • Design and implement robust data infrastructure in AWS, using Spark with Scala
  • Evolve our core data pipelines to efficiently scale for our massive growth
  • Store data in optimal engines and formats, matching your designs to our performance needs and cost factors

tvScientific is the first CTV advertising platform purpose-built for performance marketers. They leverage data and cutting-edge science to automate and optimize TV advertising to drive business outcomes. tvScientific is built by industry leaders with history in programmatic advertising, digital media, and ad verification.

US Europe

  • Design and develop scalable data pipelines and infrastructure to process large volumes of data efficiently
  • Collaborate with cross-functional teams to ensure data integrity, accessibility, and usability
  • Implement and maintain data quality measures throughout the data lifecycle

CI&T is a tech transformation specialist, uniting human expertise with AI to create scalable tech solutions. With over 8,000 employees around the world, they have a culture that values diverse identities and life experiences, fostering a diverse, inclusive, and safe work environment.

$125,000–$150,000/yr
US

  • Design, implement, and optimize robust and scalable data pipelines using SQL, Python, and cloud-based ETL tools such as Databricks.
  • Enhance our overarching data architecture strategy, assisting in decisions related to data storage, consumption, integration, and management within cloud environments.
  • Partner with data scientists, BI teams, and other engineering teams to understand and translate complex data requirements into actionable engineering solutions.

The New York Blood Center appears to be a medical organization. They are looking for a Senior Data Engineer to join their team.

$135,500–$200,000/yr
US

  • Architect, design, implement, and operate end-to-end data engineering solutions.
  • Develop and manage robust data integrations with external vendors.
  • Collaborate closely with Data Analysts, Data Scientists, DBAs, and cross-functional teams.

SmartAsset is an online destination for consumer-focused financial information and advice, helping people make smart financial decisions. With over 59 million people reached each month, they operate SmartAsset Advisor Marketing Platform (AMP) to connect consumers with fiduciary financial advisors.

$194,400–$305,500/yr
US

  • Play a Sr.tech lead & architect role to build world-class data solutions and applications that power crucial business decisions throughout the organization.
  • Enable a world-class engineering practice, drive the approach with which we use data, develop backend systems and data models to serve the needs of insights and play an active role in building Atlassian's data-driven culture.
  • Maintain a high bar for operational data quality and proactively address performance, scale, complexity and security considerations.

At Atlassian, they're motivated by a common goal: to unleash the potential of every team. Their software products help teams all over the planet and their solutions are designed for all types of work. They ensure that their products and culture continue to incorporate everyone's perspectives and experience, and never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, or marital, veteran, or disability status.

$96,050–$113,000/yr
US

  • Creating and maintaining optimal data pipeline architecture.
  • Assembling large, complex data sets that meet functional & non-functional business requirements.
  • Building the infrastructure required for optimal extraction, transformation and loading of data from a wide variety of data sources using relevant technologies.

Mercer Advisors works with families to help them amplify and simplify their financial lives through integrated financial planning, investment management, tax, estate, and insurance services. They serve over 31,300 families in more than 90 cities across the U.S. and are ranked the #1 RIA Firm in the nation by Barron’s.

$215,000–$300,000/yr
US 4w PTO 12w maternity 12w paternity

  • Design and build robust data pipelines that integrate data from diverse sources.
  • Build streaming data pipelines using Kafka and AWS services to enable real-time data processing.
  • Create and operate data services that make curated datasets accessible to internal teams and external partners.

Quanata aims to ensure a better world through context-based insurance solutions. It is a customer-centered team creating innovative technologies, digital products, and brands, backed by State Farm, blending Silicon Valley talent with insurer expertise.

$130,000–$135,000/yr
US Unlimited PTO

  • Lead the development of ETL pipelines and Data Lake infrastructure.
  • Perform ETL and Sanitation on disparate data flows from multiple internal and external sources.
  • Maintain data security, quality, and performance with details documentation.

Swing Left helps people maximize their impact on competitive elections to help Democrats win. They have one million members, who have raised more than $140 million and engaged more than 50 million voters.

$210,746–$240,000/yr
US

  • Design, build, and operate ETL pipelines at scale.
  • Design data structure for data products.
  • Develop and operate API/tools related to data products and machine learning products.

Mercari is a company that provides a marketplace platform. They value teamwork and provide career growth opportunities as the company continues to expand.

$0–$200,000/yr
North America Latin America

  • Architect and maintain robust data pipelines to transform diverse data inputs.
  • Integrate data from various sources into a unified platform.
  • Build APIs with AI assistance to enable secure access to consolidated insights.

Abusix is committed to making the internet a safer place. They are a globally distributed team that spans multiple countries and thrives in a culture rooted in trust, ownership, and collaboration.