if you are a Data Engineer with Azure and Python experience and you are looking for a now position then I have a great opportunity for you.

The project will be for 6 months +

The role can be completely remote

Position Overview: We are seeking a skilled Data Engineer with extensive experience in implementing data pipelines using Azure services, particularly Data Factory and Databricks. The ideal candidate will have a strong background in Python and PySpark, and will be adept at creating data product prototypes and supporting delivery teams with source profiling.

Key Responsibilities:

Data Pipeline Implementation:

Design and implement robust data pipelines using Azure Data Factory and other Azure services.

Utilize Azure Databricks to process and transform data efficiently.

Develop data product prototypes using Databricks to meet business requirements and support data-driven decision-making.

Provide technical support to delivery teams by conducting source profiling and ensuring data quality and integrity.

Implement Continuous Integration and Continuous Deployment (CI/CD) practices to streamline data pipeline development and deployment processes.

Project Tech Stack:

• Microsoft Azure services for data processing
• Azure Data Factory
• Azure Databricks
• Python
• PySpark

Required Skills:

Programming Languages:

Proficiency in Python and PySpark for data processing and analysis.

Hands-on experience with Azure Data Factory and Azure Databricks.

Strong understanding of data structures, algorithms, and data processing solutions.

Understanding and implementation experience of CI/CD principles in data engineering.

Qualifications:

• Proven experience in designing and implementing data pipelines using Azure services.
• Strong programming skills in Python and PySpark.
• Experience with Azure Databricks and Data Factory for data processing and transformation.
• Solid understanding of CI/CD principles and their application in data engineering projects.
• Excellent problem-solving skills and the ability to work collaboratively in a team environment.

This role is perfect for a data engineering professional who is passionate about leveraging Azure technologies to build efficient and scalable data solutions. If you have a strong background in Python, PySpark, and Azure services, and are eager to contribute to innovative data projects, we would love to hear from you.

Darwin Recruitment is acting as an Employment Business in relation to this vacancy.

Victoria Howes