Data Engineering Lead
Data Engineering Lead
We are seeking a Data Engineering Lead to oversee the design, development, and optimization of scalable data pipelines and infrastructure. This role will drive data architecture, enhance data quality, and ensure secure data management practices, while mentoring a team of data engineers. The ideal candidate has extensive experience in data engineering and leadership, with a strong technical background in big data processing, cloud solutions, and data governance.
Key Responsibilities:
- Data Architecture and Design: Develop and implement scalable data architectures, pipelines, and infrastructure for real-time and batch processing to support analytics and machine learning.
- Data Pipeline Development and Maintenance: Build and maintain robust data pipelines to ingest and transform large datasets from multiple sources; ensure quality and reliability throughout the data lifecycle.
- Collaboration and Stakeholder Engagement: Work closely with data scientists, analysts, and business stakeholders to align data solutions with business requirements and objectives.
- Team Leadership and Mentorship: Lead and mentor a team of data engineers, fostering a collaborative and innovative environment.
- Data Governance and Security: Implement and enforce data governance policies and security protocols to protect sensitive information, ensuring regulatory compliance.
- Technology Evaluation and Innovation: Stay updated on industry trends and emerging technologies, recommending tools and practices that enhance data infrastructure and team capabilities.
Requirements:
- Education: Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or related field.
- Experience: 8+ years in data engineering, with 2+ years in a leadership role.
- Technical Skills:
- Proficiency in data lake/data warehousing solutions (e.g., Snowflake, Databricks) and big data frameworks (e.g., Hadoop, Spark).
- Experience with cloud platforms (e.g., AWS, Azure) and data orchestration tools (e.g., Airflow, Dagster).
- Strong SQL skills and expertise in database technologies (e.g., Postgres, SQL Server).
- Knowledge of ETL tools (e.g., Talend, Informatica) and data governance frameworks.
- Programming experience with Python, Java, or Scala for data manipulation; familiarity with machine learning infrastructure and MLOps a plus.
- Experience with BI tools (e.g., Tableau, Power BI).
Soft Skills:
- Effective leadership and communication abilities.
- Strong problem-solving skills and attention to detail.
- Ability to manage multiple projects in a dynamic environment.
This role provides an opportunity for a senior data engineering professional to lead a team in building a high-quality, scalable data infrastructure that aligns with business goals.