Data Engineer

Data Engineer :

On behalf of our Oil & Gas client, Procom is searching for a Data Engineer for a 3-month role with potential extensions. This position is a onsite at our client’s Calgary office.

Data Engineer - Job Description :

This project involves upgrading all ETLs by transitioning Informatica-mediated data loading workflows to Azure Data Factory and Azure Databricks. This upgrade will facilitate the IDB migration to an Azure SQL Managed Instance.

Data Engineer - Responsibilities :

• Work closely with the Practice Lead – Data Engineering to design, implement, and deploy data transformation and consolidation pipelines.
• Analyze and understand existing Informatica ETL code, then design and rewrite it using Azure Data Factory and Databricks.
• Collaborate with Subject Matter Experts and Business Analysts to understand rewrite requirements and populate data schemas.
• Document design and architecture as needed.
• Conduct testing of data pipelines and support user acceptance testing cycles.
• Combine data from multiple sources to build the target consolidated state, documenting source-to-target mappings.
• Identify and document data dependencies.
• Engage in agile delivery practices, participate in sprint planning, estimation, and retrospectives.

Data Engineer - Mandatory Skills :

• Experienced in developing data engineering pipelines using Azure Data Factory, MS-Azure, and Azure Databricks.
• Proficient in designing and developing data engineering components of cloud data architecture in MS Azure.
• Knowledge of Enterprise Cloud Data Architecture in Microsoft Azure.
• Ability to create and test DDL and DML scripts.
• Python scripting experience.
• Experience with Azure DevOps.
• Strong communication and teamwork skills.

Data Engineer – Nice-to-Have Skills :

• Ability to provide recommendations to support data strategy.
• Experience with enterprise data migration to cloud-based technologies.
• Skilled in managing deployable data pipelines in Microsoft Azure environment.
• Informatica PowerCenter experience.

Data Engineer – Assignment Length :

This is a 3-month contract position with potential extensions

Data Engineer - Assignment Location :

Calgary, Alberta, Canada.