Talent.com
Data Engineer

Data Engineer

CB CanadaNorth Vancouver, British Columbia, Canada
30+ days ago
Salary
CA$80,000.00 yearly
Job description

Data Engineer

On behalf of our client, Procom Consultants Group is currently looking for an Intermediate Data Engineer.

Requested resources to provide developer expertise on coding of data transformation pipelines and systems integration tasks relating to enterprise data management applications that span across different business areas and platforms.

Data Engineer Job Details

  • Creating mapping documentation of data elements from source to target and liaising with other Data Engineers, Data Architects and SMEs.
  • Developing & testing Data Transformation pipelines by leveraging the latest Big Data tools and technologies
  • Collaborate with upstream data teams, develop data validation and test plans, performing user acceptance testing, and provide feedback to development and sustainment teams
  • Conducting analysis for moderate to complex data requests, defining data fields and determining data availability, developing information layout, format and interactivity. Presenting findings and providing clarification.
  • Act as a lead developer to implement code changes in various environments and platforms leveraging our DevOps & DataOps
  • Work closely with the testing team to assist with issues raised in user acceptance testing (UAT) and Business Value Testing (BVT)

Deliverables :

Fully implement the new data format across all IM supported platforms including

Big Data (Hadoop), Enterprise Data Warehouse (EDW) and Operational Data Store

ODS)

Data Engineer Mandatory Skills

Communication & Problem Solving Skills :

  • Communicating & presenting solutions to internal developers
  • Ability to apply critical thinking skills to develop efficient code, data transformation logic and solution design
  • Experience and / or understanding of Agile Methodologies

    • Technical Skills :

  • Advanced Database Skills & SQL
  • Experience with or exposure to Big Data platforms, ideally exposure to Hadoop ecosystem (HDFS, Apache Hive, Apache Spark, SparkSQL)
  • Extensive programming experience in either Java, C++ or preferably Scala, IntelliJ, Jenkins
  • Working experience with Apache Spark Data Processing Framework, Parquet data format
  • Good understanding of Relational (Oracle / DB2) and Hadoop Big Data platforms
  • Experience working with Jira

    • Data Engineer Start Date

    Oct 24, 2022

    Data Engineer Assignment Duration

    March 31 2023

    Data Engineer  Working Location

    Fully remote but PST hours