Data Engineer

Stanowisko: Data Engineer

Miejsce pracy: Warszawa

Data dodania: 24.02.2021

Diverse CG is a modern technology company, gathering in its ranks IT related professionals. Due to the continuous development and the large number of recruitment projects that we carry out for our Partners, we are looking for a person for the position:

Data Engineer

Responsibilities:

  • Port the existing use cases and processes from our legacy hadoop-based on premise plaftorm to the Azure Cloud, using state-of-the art technologies (Databricks, Spark 3, Azure Cognitive Services, Azure Event Hub, Docker, Azure Pipelines, Azure Data Factory, Scala, Python)
  • Help with the onboarding of new project squads directly on the new cloud Data Platform
  • Gather engineering requirements and patterns common to data use cases and data squads
  • Design, build and maintain common building such as CI/CD Pipelines, shared libraries (data pipeline development, data quality, data lineage) and shared services (REST API, data viz, monitoring, scheduler)
  • Support a community of data engineers and data scientists by understanding their problems and answering their questions and help them write the solutions on the Data Platform
  • Participate to the build of our Data Science platform
  • Participate to the data onboarding of third-party data providers such as Bloomberg or internal applications
  • Design and maintain APIs
  • Build a research environment for our Quants

Requirements:

  • Spark (preferably on Databricks)
  • Scala or Python (preferably both)
  • Cloud computing practices: Infra as Code, security (preferably on Azure)
  • Experience working with data of various formats (np. Avro, Parquet, JSON, CSV)
  • Git + CD/CI
  • Experience in designing and building APIs + API management is a plus
  • Azure Cloud – Kubernetes, DataFactory, Azure functions, Cognitive Services, Event Hub, Pureview, Webapp
  • Docker
  • Azure DevOps
  • Data Bitemporal experience

Offer:

  • Private medical care
  • Various forms of employment
  • Constant support of dedicated consultant
  • Team-building events organised by DCG
  • Competitive salary adequate to experience and skills

Przed wyslaniem CV/dokumentów aplikacyjnych, uprzejmie prosimy kandydatów o zapoznanie się z klauzulą informacyjną.

Po zapoznaniu się z klauzulą informacyjną podaną przez Administratora Danych dobrowolnie wyrażam zgodę na przetwarzanie przez Ogłoszeniodawcę moich danych osobowych, zawartych w mojej ofercie pracy, dla potrzeb procesu tej rekrutacji oraz dla realizacji przyszłych projektów rekrutacyjnych.

Agencja doradztwa personalnego (nr licencji 4642)

Dane spółki: Diverse CG Sp. z o.o. Sp.k., ul. Towarowa 28, 00-839 Warszawa                                                     

REGON: 141316780

NIP: 5222877930

KRS: 0000733458

Obserwuj nas: