Implement and optimize scalable, reliable, maintainable, and high-performance data pipelines for extraction, loading, transformation, and orchestration.
Apply software engineering practices including CI/CD, infrastructure-as-code, containerization, and cloud-native deployments.
Collaborate with business stakeholders to turn use cases into production-ready services and solutions, owning systems from concept through production.
Implement testing and monitoring practices to maintain data quality and integrity.
Requirements
Bachelor’s degree or higher in a STEM field is required.
A concentration in Computer Science, Math, Physics, or another engineering-related field is preferred.
At least 5 years of experience in data engineering or a related discipline is required.
Expertise in Python and SQL, including data manipulation and analysis, is required.
Proficiency with Databricks/PySpark and dbt for data warehousing and transformation is required.
Experience with workflow orchestration tools such as Airflow or Dagster is required.
Demonstrated experience designing complex data systems from the ground up and handling large-scale data projects, including data cleaning, ETL, and information retrieval.
Experience with LLMs, prompt engineering, RAG, vector databases, machine learning principles, feature engineering, or knowledge graphs is a plus.
Strong verbal and written communication skills are required.