W1siziisijiwmjavmduvmdqvmtavmjivmjevmziwl3dlymluyxigagvhzgvylmpwzyjdlfsiccisinrodw1iiiwimtawmhg5mdbcdtawm2mixv0

I AM A

JOBSEEKER.

continue to candidate homepage

W1siziisijiwmtkvmdkvmjcvmtmvntkvndmvnju3l1jnmv8ynzg5ifttbwfsbf0uanbnil0swyjwiiwidgh1bwiilcixmdawedkwmfx1mdazyyjdxq

I AM

HIRING.

Continue to client homepage

Data Engineer

  • Location

    New York, USA

  • Sector:

    Data Engineering

  • Job type:

    Permanent

  • Salary:

    Negotiable

  • Contact:

    Callum Cobb

  • Email:

    callum.cobb@darwinrecruitment.com

  • Job ref:

    JN -062021-88701_1624366903

  • Published:

    4 months ago

  • Expiry date:

    2021-09-20

Responsibilities:

  • Work within a cross-functional team of engineers to build infrastructure for a data processing ecosystem supporting real-time and batch data that underpins all analytic and operational data flows
  • Build tools and services to support data discovery, lineage, governance, and privacy compliance across the data platform
  • Build and maintain a CICD process for infrastructure components to ensure high availability while remaining nimble
  • Leadership and responsibility for technical architecture, reliability, resilience, and scalability of the data platform
  • Influence and drive data engineering and architecture best practices and standards within the team and wider organization, along with a culture of quality, innovation, and experimentation
  • Work with other teams: evangelize the platform, best-practices, data driven decisions; identify new use cases and features and drive adoption
  • Work with and manage a small team of remote resources working in a different time zone

Qualifications:

  • You're a self-learner, team player, and have excellent problem-solving skills.
  • Want to help other teammates grow through mentorship and coaching.
  • Knowledge of design patterns and data engineering best practices
  • Strong programming experience in SQL.
  • Developing and maintaining ETL/ELT pipeline with Python, Spark, Java programming languages. Experience with ETL tools such as Matillion, Dell Boomi, Luigi, Informatica, Talend, and or SSIS.
  • A deep understanding of relational and MPP database technology, working with at least one of the major databases platforms (Oracle, SQLServer, Teradata, MySQL, Postgres) and cloud based MPP databases (Redshift, Synapse, Big Query, Snowflake)
  • Professional development experience with some of the AWS/Azure/GCP data stack to deploy data pipelines e.g., S3, Blob storage, AWS Glue, Azure Data Factory, Cloud Dataflow, EMR, Dataproc, etc

Darwin Recruitment is acting as an Employment Agency in relation to this vacancy.