Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Lead Data Engineer - GCP

Home > Python programming jobs

Lead Data Engineer - GCP in Others new

  • Hyderābād, India

Responsibilities

  • Gather customer requirements and participate in discovery, solution design, and architecture activities.
  • Design, develop, test, and maintain scalable Python-based ETL and ELT data pipelines.
  • Architect enterprise data solutions using GCP services, Kafka, GKE, Apigee, dbt, and related technologies.
  • Implement data ingestion, transformation, cleansing, validation, monitoring, and data quality controls.
  • Use Dataflow, Cloud Functions, BigQuery, Cloud Composer, Google Cloud Storage, IAM, and Cloud Run for data engineering solutions.
  • Collaborate with data scientists, analysts, and engineering teams to deliver efficient data solutions.
  • Manage version control with GitHub and participate in data project deployment pipelines.
  • Write complex SQL queries for extraction and validation from SQL Server, Oracle, and PostgreSQL databases.
  • Document pipeline designs, data flow diagrams, and operational support procedures.
  • Provide post-delivery customer support.

Requirements

  • 10+ years of hands-on Python experience in backend or data engineering projects.
  • Strong working experience with Google Cloud Platform, especially Dataflow, BigQuery, Cloud Functions, and Cloud Composer.
  • Strong understanding of data pipeline architecture, data integration, and transformation techniques.
  • Experience with Apache Spark, PySpark, Kafka, Redis, FastAPI, Airflow, and GCP Composer DAGs.
  • Strong SQL experience with at least one enterprise database such as SQL Server, Oracle, or PostgreSQL.
  • Experience with GitHub and CI/CD practices.
  • Experience migrating data from on-premise sources to cloud platforms.
  • Bachelor's degree in Computer Science or a related field, or equivalent experience.
  • Experience with Snowflake, Databricks, Azure Data Factory, other Azure data engineering tools, or AWS services is optional.
  • Strong problem-solving, analytical, communication, and collaboration skills.

Egen

Egen is a technology services company with leading capabilities in cloud, data engineering, analytics, AI, and platform engineering. Our more than 700 technology specialists in Canada, Europe, India and the United States have deep expertise and experience working with many of the cloud and data p...

Similar positions

Mid-Level AI Engineer

  • Clera
  • Full time
  • USA
  • 11/22/2026
  • Salary: $150k - $180k/yr
  • San Francisco, CA

Site Reliability Engineering Professional

  • BT
  • Full time
  • UK
  • 11/22/2026
  • London

Software Engineering Technical Leader

  • Cisco
  • Full time
  • Others
  • 11/22/2026
  • Bengaluru, India

Bitcoin Security Engineer

  • MicroStrategy
  • Full time
  • USA
  • 11/22/2026
  • Salary: $104k - $187k/yr
  • Tysons, VA

Full-Stack AI Engineer

  • Pavago
  • Full time
  • Remote
  • 11/22/2026
  • Portugal