Lead Data Engineer Job at Collective Health, San Francisco, CA

dnFJdmZobzNuZHp1akh4bk1tTkpBbzY0V2c9PQ==
  • Collective Health
  • San Francisco, CA

Job Description


At Collective Health, we’re transforming how employers and their people engage with their health benefits by seamlessly integrating cutting-edge technology, compassionate service, and world-class user experience design.

We deliver a connected healthcare experience for over 600,000 members and 70+ companies nationwide who want the best for their employees. Our data engineering team is pivotal in this mission, constructing and managing scalable, high-performance data pipelines and architectures that drive analytics, operational workflows, and machine learning applications.

As a Lead Data Engineer, you will drive the development of robust, scalable, and efficient data solutions, collaborating closely with cross-functional teams. You will provide thought leadership on data architecture, mentor junior engineers, and optimize our data ecosystem for performance and reliability.

What you'll do:



  • Architect Scalable Data Solutions - Design, develop, and optimize large-scale data pipelines using Spark (PySpark, Scala), Databricks, and distributed data processing frameworks.

  • Advance Data Modeling & Architecture - Lead the design and evolution of data models to support analytical, operational, and machine-learning requirements.

  • Enhance Data Performance & Reliability - Improve data processing performance, scalability, and reliability, while ensuring data quality and governance.

  • Drive Cross-Functional Collaboration - Partner with Product, Engineering, Data Science, and Analytics teams to deliver high-impact data solutions that generate actionable business and clinical insights.

  • Mentor & Provide Technical Leadership - Guide junior and mid-level engineers, conduct code reviews, and establish best practices in data engineering.

  • Ensure Data Governance & Security - Implement robust security, privacy, and compliance measures for sensitive healthcare data, ensuring adherence to industry regulations.

  • Influence Data Strategy - Provide input on data infrastructure decisions, emerging technologies, and process improvements.

To be successful in this role, you'll need:



  • 8+ years of data engineering experience in fast-paced, data-driven environments.

  • Expertise in building scalable ETL pipelines with Spark (PySpark or Scala) and SQL.

  • Deep understanding of data architecture, schema design, and dimensional modeling for analytics and machine learning.

  • Proficiency in distributed systems such as Spark, Databricks, or Snowflake.

  • Experience with event-driven architectures and streaming platforms like Kafka or Kinesis.

  • Excellent communication skills – ability to collaborate cross-functionally and translate complex technical concepts into business impact.

  • Mentorship experience – experience guiding engineers and fostering a collaborative, inclusive team culture.

  • Security-first mindset – familiarity with data privacy, encryption, and compliance in healthcare or other regulated industries is a plus.

Pay Transparency Statement 


This is a hybrid position based out of one of our offices: San Francisco, CA, Plano, TX, or Lehi, UT. Hybrid employees are expected to be in the office two days per week. #LI-hybrid  

The actual pay rate offered within the range will depend on factors including geographic location, qualifications, experience, and internal equity. In addition to the salary, you will be eligible for stock options and benefits like health insurance, 401k, and paid time off. Learn more about our benefits at .

San Francisco, CA Pay Range

$168,000—$210,000 USD

Lehi, UT Pay Range

$134,500—$168,000 USD

Plano, TX Pay Range

$147,800—$185,500 USD

Why Join Us?



  • Mission-driven culture that values innovation, collaboration, and a commitment to excellence in healthcare

  • Impactful projects that shape the future of our organization

  • Opportunities for professional development through internal mobility opportunities, mentorship programs, and courses tailored to your interests

  • Flexible work arrangements and a supportive work-life balance

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. Collective Health is committed to providing support to candidates who require reasonable accommodation during the interview process. If you need assistance, please contact recruiting-accommodations@collectivehealth.com .

Privacy Notice


For more information about why we need your data and how we use it, please see our privacy policy: .

Job Tags

Full time, Work at office, Flexible hours, 2 days per week,

Similar Jobs

Winters Independent School District

Computer Support Technician Job at Winters Independent School District

 ...Position Type: Technology/Computer Technician Date Posted: 7/11/2024 Location: Winters ISD Computer Support Technician...  ...meeting the technology needs of the district; troubleshooting and repairing computer hardware and software, and resolving immediate... 

Best Buy

Warehouse Merchandise Processor Job at Best Buy

 ...Warehouse Merchandise Processor at Best Buy summary: The Warehouse Merchandise Processor is responsible for processing, inspecting...  ...designated areas Fulfill online orders by locating, packing and shipping products to their destinations Assess visible condition and... 

AllStar Wings & Ribs Vaughan

Restaurant Server Job at AllStar Wings & Ribs Vaughan

 ...Job Description Job Description We are looking for enthusiastic and energetic servers to join our team at AllStar Wings & Ribs Corporate Flagship Location in Vaughan, Ontario. Must be able to multitask and have excellent customer service skills while displaying a pleasant... 

Experis

Senior Accountant Job at Experis

 ...Jefferson Wells is proud to partner with a premier client in search of a skilled Senior Accountant to step in and provide critical leadership and support during a high-priority period. ~ Location: Green Bay Full Time On-Site***Flexibility on schedule and/or... 

Actalent

Pharmacist Job at Actalent

Clinical Review Pharmacist (100% Remote, Entry-Level)Pharmacist License RequiredJob Description...  ...and quantity limit exceptions. You will work independently within a team of pharmacy...  ...This is a fully remote work-from-home position, available in any state except California...