Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Sr Data Engineer

Home > Python programming jobs

Sr Data Engineer in Remote

  • Capgemini
  • Full time
  • Email
  • Mexico

Responsibilities

  • Build and optimize Spark pipelines on Microsoft Fabric or Databricks.
  • Diagnose performance using Spark UI and Delta statistics.
  • Manage Delta Lake design, Fabric capacity consumption, compute sizing, throttling exposure, and operational reliability.
  • Create reusable Python libraries and Spark modules for ingestion, governance, security, monitoring, and optimization.
  • Troubleshoot Spark skew, spills, out-of-memory errors, task execution, and query performance.
  • Optimize Delta Lake through partitioning, liquid clustering, Z-Order, OPTIMIZE, VACUUM, and MERGE strategies.
  • Design, optimize, and troubleshoot large-scale data pipelines.
  • Build production-grade data solutions with testing, logging, monitoring, and documentation.

Requirements

  • At least 5 years of experience working as a data engineer, ETL engineer, or in similar roles.
  • Strong production experience with Microsoft Fabric or Databricks, including performance tuning and operational support.
  • Experience troubleshooting Apache Spark performance issues, including skew, spills, out-of-memory errors, task execution, and query optimization.
  • Hands-on experience with Delta Lake optimization techniques, including partitioning, liquid clustering, Z-Order, OPTIMIZE, VACUUM, and MERGE.
  • Strong SQL and data engineering skills for large-scale pipeline design and troubleshooting.
  • Python development experience building reliable, production-grade data solutions.
  • Fluent English proficiency.

Benefits

  • Work on meaningful technology projects addressing real-world challenges.
  • Join a company focused on sustainability, inclusion, and digital equity.
  • Contribute to initiatives supporting people, planet, and society.
  • Work remotely from any state in Mexico.

Capgemini

Capgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organizations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of 420,...

Similar positions

Lead Data Engineer (P3999)

  • 84.51˚
  • Full time
  • USA
  • 11/18/2026
  • Salary: $125k - $207k/yr
  • Chicago, IL

Software Engineer, Datacenter Server Lifecycle

  • Anthropic
  • Full time
  • USA
  • 11/18/2026
  • Salary: $405k - $485k/yr
  • Remote

Lead AI Engineer

  • ALO
  • Full time
  • USA
  • 11/18/2026
  • Salary: $180k - $220k/yr
  • Beverly Hills, CA

Staff Integration Engineer

  • Zscaler
  • Full time
  • Remote
  • 11/18/2026
  • India

Enterprise Security Engineer (Okta / Identity)

  • Socure
  • Full time
  • USA
  • 11/18/2026
  • Salary: $165k - $230k/yr
  • New York, NY