Cloud Data Engineer

Stanowisko: Cloud Data Engineer

Miejsce pracy: 100% remote

Data dodania: 07.10.2024

As a recruitment company, DCG understands that every business is powered by experienced professionals. Our management style and partnership approach enable us to meet your needs and provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for:

Cloud Data Engineer

Responsibilities:

  • Implement and manage data ingestion pipelines from diverse sources such as Kafka, RDBMS (Postgres) using CDC (Change Data Capture), and file systems (CSV) following Medalion Architecture principles
  • Develop and optimize data transformations using PySpark and SQL to handle data ranging from MB to GB, depending on the source
  • Conduct unit testing and integration testing to ensure the accuracy and reliability of data transformations and pipelines
  • Work with AWS technologies, including S3 for data storage and Docker on AWS for containerized applications
  • Implement and manage infrastructure using Terraform, such as creating S3 buckets, managing Databricks Service Principals, and deploying infrastructure as code
  • Deploy and manage solutions using CI/CD pipelines, particularly with CircleCI, to ensure seamless and automated deployment processes

 

Requirements:

  • Minimum 4-5 years of a professional experience
  • Proficiency in SQL and Python
  • Strong experience with AWS cloud services
  • Hands-on experience with DataBricks
  • Knowledge of ETL Processing
  • Effective communication skills in English (minimum B2 level)
  • Knowledge of system design
  • Understanding of Medalion Architecture

Nice to have:

  • Familiarity with Kedro and Airbyte 
  • Knowledge of Machine Learning

 

Offer:

  • Private medical care
  • Co-financing for the sport card
  • Training & learning opportunities
  • Constant support of dedicated consultant
  • Team-building events organized by DCG
  • Employee referral program

Przed wyslaniem CV/dokumentów aplikacyjnych, uprzejmie prosimy kandydatów o zapoznanie się z klauzulą informacyjną.

Po zapoznaniu się z klauzulą informacyjną podaną przez Administratora Danych dobrowolnie wyrażam zgodę na przetwarzanie przez Ogłoszeniodawcę moich danych osobowych, zawartych w mojej ofercie pracy, dla potrzeb procesu tej rekrutacji oraz dla realizacji przyszłych projektów rekrutacyjnych.

Agencja doradztwa personalnego (nr licencji 4642)

Dane spółki: DCG Sp. z o.o., ul. Towarowa 28, 00-839 Warszawa                                                     

REGON: 141316780

NIP: 5222877930

KRS: 0001067305

Obserwuj nas:
UE