Position : Apache Airflow Data Engineer
Client : Financial Services Client
Location : Toronto Hybrid
Duration : 3 Months initial contract
Security Clearance : N / A
Language : English
Background :
Kyndryl Canada is supporting our Financial Services client and their project to build and maintain ETL pipelines to process and move data more efficiently.
Qualifications :
- Extensive experience working with Apache Airflow.
- Experience working with data process automation.
- Ability to work on-site in downtown Toronto, multiple days a week.
Responsibilities could entail :
- Data Pipeline Development : Build and maintain ETL pipelines to process and move data efficiently.
- Workflow Orchestration : Use Apache Airflow to schedule and manage complex data workflows.
- Automation : Automate data processing tasks to ensure they run reliably and on time.
- Performance Optimization : Optimize data pipelines for speed and efficiency.
- Monitoring and Logging : Monitor pipeline performance and maintain detailed logs for troubleshooting.
- Collaboration : Work with data scientists and analysts to meet data needs.
Data Engineer (Airflow)
Data Pipeline Development: Build and maintain ETL pipelines to process and move data efficiently. Workflow Orchestration: Use Apache Airflow to schedule and manage complex data workflows. Collaboration: Work with data scientists and analysts to meet data needs. Kyndryl Canada is supporting our Finan...
Data Engineer (Vertica OR Snowflake & Airflow OR Spark)
Familiarity with data pipelines that use Airflow and Spark or equivalent tools. On-board NRT data pipeline as per business requirement Performance tuning of bad performing queries and data model redesign. Experience with OLAP databases such as Vertica and or Snowflake. Creating Vertica Snowflake dat...
Data Engineer with Python
Role: Python Data Engineer(Databricks/Pyspark, SQL, Azure). ETL developers are responsible for designing and creating the data warehouse and all related extraction, transformation and load of data functions. Strong understanding or Snowflake on Azure Architecture, design, implementation and operatio...
AWS DevOps Engineer
As a DevOps engineer setup CI/CD pipeline for application deployment. Work in the DevOps team, to build new shared infrastructure services for on premises failover environment: S3, Kafka, Data Store. Work with DevOps team to establish connectivity between new failover shared services and existing sh...
Data Engineer
Hybrid Hadoop Engineer and Hadoop Infrastructure Administrator to build and maintain a scalable and resilient Big Data framework to support Data Scientists. Ability to provide quick ingestion tools and corresponding access APIs for continuously changing data schema, working closely with Data Enginee...
Lead Data Engineer
Integrate data from various resources and manage big data:Collect and consolidate data from external sourcesHow to thrive in this role?10 years of experience in a Data Engineer / Data Specialist role. Good knowledge of Azure data services (Azure Data Factory, Synapse, Azure Data Lake Storage, Event ...
Data Science Engineer
As a data scientist you will become intimately familiar with the data and structure of the graph to product valuable insights and predictive models for all Trials. Our teams work on multiple projects simultaneously, leveraging advanced data analytics and problem-solving techniques. We are looking to...
DevOps Engineer
Agile Methodology, Azure Kubernetes Service (AKS), Cloud Architectures, Cloud Infrastructure, Cloud Security, DevOps, Information Technology (IT) Infrastructure, Jenkins Continuous Integration, Linux, Microsoft Azure, Programming Languages, Public Clouds, Python (Programming Language), Software Chan...
Lead Data Engineer
Lead an engineering team to design, code, and test cloud-based data pipelines to process and store large-scale datasets. Ipsos Global Data Management Tower serves as the backbone of Ipsos’ efforts to harness and integrate the power of data, globally. We’re looking for someone with a deep interest an...
Junior Data Engineer
We are seeking a Junior Data Engineer with an analytical mindset and a passion for using data to drive decision-making. As a member of our team, you will play a vital role in developing and interpreting data insights, as well as building and maintaining data pipelines and dashboards. Design, develop...