Database Administrator/Engineer
Database Administrator/Engineer
Job Summary: The Intersect Group is seeking a Senior Database Administrator/Engineer for our direct client, where you will handle the development of all aspects of data processing, including database management, architecture/modeling, and ETL processing. Create and deliver analytical solutions using various tools. Collaborate with corporate data analytics team and business stakeholders to collect requirements and translate them into technical specifications and process documentation. Work on the architecture and development of an event-driven data warehouse, streaming, batch, data modeling, and storage. Provide technical guidance. Review code and test results. Review pulls requests and scripts for production development. Work on advanced databases, to write and optimize SQL queries, stored procedures, functions, partition data, indexing, and reading execution plans. Write and debug Python/PySpark scripts to generate extracts, clean, conform, and deliver data for consumption. Implement ETL architecture, data profiling, process flow, metric logic, and error handling. Support continuous improvement by exploring and presenting alternatives to processes and technologies.
Responsibilities and Duties:
- Enable the design of conceptual architecture and technical solutions.
- Design, develop and maintain scalable and efficient data pipelines and workflows using Azure Synapse or like tools to support various business use cases and analytical requirements.
- Collaborate closely with cross-functional teams to understand data requirements and translate them into technical solutions.
- Implement and manage CI/CD pipelines using Azure DevOps to automate the deployment, testing and monitoring of data engineering solutions, ensure continuous integration and delivery.
- Optimize and tune data pipelines and process workflows for performance, scalability, and reliability.
- Collaborate with data architects, engineers, and analysts to design, develop, and maintain data pipelines and ETL processes.
- Leverage best practices and monitoring tools to identify and address bottlenecks and inefficiencies.
- Utilize Azure Data Studio to design and optimize database schemas, stored procedures, and SQL queries.
- Ensure best practices in data integrity, efficiency, and security.
- Monitor and troubleshoot data pipeline health and performance issues, proactively identifying and resolving issues to minimize downtime and ensure data quality and reliability.
- Create and maintain comprehensive documentation for data pipelines, workflows, and system configurations, ensuring knowledge transfer and compliance with organizational standards and policies.
- Conduct data profiling, cleansing and validation activities to ensure data quality and integrity throughout the data life cycle, adhering to data governance and compliance requirements.
- Participate in code review, testing, and debugging activities to ensure the reliability, stability, and maintainability of data engineering solutions.
- Provide technical guidance, mentorship, and support to team members.
- Troubleshoot and resolve data-related issues reported by end-users or detected through monitoring and alerting systems, ensuring timely resolution and minimal disruption to operations.
- Design and implement a mature data lifecycle, including backups and restore of data.
- Assist ServiceNow admins with creating workflows relevant to the data lifecycle.
- Ensure compliance with Role-Based Access and other regulatory requirements.
- Developing and implementing data governance policies and procedures to ensure data accuracy, completeness, and consistency.
- Ensuring data quality and integrity by implementing data validation, cleansing, and testing methods
- Troubleshooting and resolving data issues and providing technical support as needed
- 7+ years experience in Data Engineering in a Microsoft environment required.
- Bachelor's degree required.
*Must be eligible to work for any U.S. employer without sponsorship