Data Engineer

Stanowisko: Data Engineer

Miejsce pracy: 100% remote

Data dodania: 06.09.2024

As a recruitment company, DCG understands that every business is powered by experienced professionals. Our management style and partnership approach enable us to meet your needs and provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for:

Data Engineer

Responsibilities:

  • Design, manage, and optimize ETL processes using Azure Data Factory for batch data processing
  • Use Databricks and PySpark to process and analyze large datasets across distributed environments
  • Create automation workflows and perform complex data manipulations using Python
  • Write and optimize SQL queries to efficiently retrieve and process data from various sources
  • Implement real-time data ingestion solutions using Azure Event Hub and Spark Streaming
  • Manage and interact with NoSQL databases, particularly CosmosDB, to support scalable data storage and retrieval
  • Prioritize and enhance batch data processing workflows, ensuring high performance and reliability
  • Collaborate with cross-functional teams, provide insights, and maintain thorough documentation of all data processes 

 

Requirements:

  • Minimum 3 years of experience in similar position
  • English level: B2, C1
  • Strong experience with Azure Data Factory for orchestrating ETL processes
  • Proficiency in Databricks and PySpark for distributed data processing
  • Advanced skills in Python for data manipulation and automation
  • Solid knowledge of SQL for writing complex queries and optimizing data retrieval
  • Experience with Azure Event Hub for real-time data ingestion
  • Knowledge of CosmosDB for working with NoSQL databases
  • Understanding of Spark Streaming for processing real-time data streams
  • Batch processing experience is more critical than streaming processing for this role
  • Familiarity with Azure Synapse is a plus, but not mandatory
  • Good to Have: Knowledge of Azure DevOps for CI/CD pipeline automation, experience in the Aviation Industry, and familiarity with Copilot as a tool for AI-driven development assistance

 

Offer:

  • Private medical care
  • Co-financing for the sport card
  • Training & learning opportunities
  • Constant support of dedicated consultant
  • Team-building events organized by DCG
  • Employee referral program

Przed wyslaniem CV/dokumentów aplikacyjnych, uprzejmie prosimy kandydatów o zapoznanie się z klauzulą informacyjną.

Po zapoznaniu się z klauzulą informacyjną podaną przez Administratora Danych dobrowolnie wyrażam zgodę na przetwarzanie przez Ogłoszeniodawcę moich danych osobowych, zawartych w mojej ofercie pracy, dla potrzeb procesu tej rekrutacji oraz dla realizacji przyszłych projektów rekrutacyjnych.

Agencja doradztwa personalnego (nr licencji 4642)

Dane spółki: DCG Sp. z o.o., ul. Towarowa 28, 00-839 Warszawa                                                     

REGON: 141316780

NIP: 5222877930

KRS: 0001067305

Obserwuj nas:
UE