Similar Jobs
See allSenior Data Engineer
Mobile Wave Solutions
Databricks
PySpark
Delta Lake
Senior Data Engineer - Databricks
SugarAI
US
Databricks
PySpark
SQL
Senior Data Engineer (Databricks)
Massive Rocket
UK
Databricks
Python
SQL
Data Engineer
Ardent
US
SQL
Python
Databricks
Data Engineer (Databricks)
Livefront
Peru
Databricks
Python
SQL
Responsibilities:
- Design and build ingestion pipelines, batch and streaming where appropriate, from enterprise source systems into the Databricks lakehouse using Delta Lake.
- Own Bronze-layer ingestion, including raw landing patterns, metadata capture, load traceability, and recoverable ingestion design.
- Build Silver-layer pipelines for cleansing, standardization, deduplication, conformance, and quality enforcement without embedding unauthorized or undocumented KPI logic.
Qualifications:
- 12+ years of data engineering experience, including hands-on ownership of production data pipelines.
- Strong Databricks experience, including Delta Lake, Databricks Workflows or Jobs, and Spark with PySpark and/or Spark SQL.
- Working knowledge of Unity Catalog, including catalogs, schemas, tables, lineage, and access control concepts.
Preferred Qualifications:
- Databricks certification such as Data Engineer Associate or Professional.
- Experience standing up or maturing Unity Catalog structures and access models in an enterprise setting.
- Experience in defense, aerospace, financial services, healthcare, or another highly regulated environment.
Shield AI
Shield AI is a venture-backed defense-tech company founded in 2015 with the mission of protecting service members and civilians with intelligent systems, developing products like Hivemind autonomy software and V-BAT and X-BAT aircraft. The company has offices and facilities across the U.S., Europe, the Middle East, and Asia-Pacific, and its technology actively supports operations worldwide.