Search jobs > Toronto, ON > Data engineer

Data Engineer - Toronto, ON

AIT Global
Toronto, Ontario, Canada
$102.2K-$142K a year (estimated)
Full-time

Job Title : Data Engineer

Location : Toronto, ON

Skills : Data Engineering · Data Analytics · Git · Amazon Web Services (AWS) · Unix · MySQL · Amazon Redshift · Python (Programming Language), Machine Learning, Oracle DB.

  • Develop python scripts that report status of different workflows helping monitor batch runs effectively.
  • Add logging handlers to give custom log messages, improving debugging efficiency and understand code flow.
  • Optimize service executions, end to end data flow using hold functionality, priority and parallel task execution.
  • Create an emailer framework to notify about important run information, server issues and sending data files.
  • Enhance multiple framework by adding rerun checks, try except blocks, building idempotent data pipelines.
  • Improve performance of difference tasks by reducing the iterations to fetch data and tuning the sql queries.
  • Capable to oversee an offshore team of Data Engineer, Tester, Production Support.
  • Develop python scripts that report status of different workflows helping monitor batch runs effectively .
  • Add logging handlers to give custom log messages, improving debugging efficiency and understand code flow.
  • Optimize service executions, end to end data flow using hold functionality, priority and parallel task execution.
  • Create an emailer framework to notify about important run information, server issues and sending data files.
  • Enhance multiple framework by adding rerun checks, try except blocks, building idempotent data pipelines.
  • Improve performance of difference tasks by reducing the iterations to fetch data and tuning the sql queries.
  • 30+ days ago
Related jobs
AIT Global
Toronto, Ontario

Data Engineering · Data Analytics · Git · Amazon Web Services (AWS) · Unix · MySQL · Amazon Redshift · Python (Programming Language), Machine Learning, Oracle DB. Optimize service executions, end to end data flow using hold functionality, priority and parallel task execution. Optimize service execut...

University of Toronto
Toronto, Ontario

Under the general direction of the Executive Director, Enterprise Applications & Solutions Integration or their designate, you will be part of a dynamic, collaborative and innovative team focused on delivering transformational technology solutions to the University. The Enterprise Applications a...

S.i. Systems
Toronto, Ontario

The Business Information Management Specialist provides business technical leadership across a broad range of information management functions to support the various areas of data and analytics. Work Location (Remote, Hybrid or Both?): Hybrid, onsite twice a week in Toronto. Works independently as a...

Simple Solutions
Toronto, Ontario

SAP BTP Techno-functional Engineer Consultants - hybrid on site Austin . Core Data Services (CDS) with Strong Cloud Application Programing model(CAPM), OData & JavaScript Experience. As SAP BTP Engineer, you will be responsible for designing, developing, and deploying cloud enterprise solutions usin...

Marketing
Ontario

With an increased focus on digital transformation, FCTG is seeking a Marketing Data and Technology Leader to implement data and technology solutions, and lead enablement and adoption by our marketing teams. This role will be the leader in a Marketing Technology, including but not limited to: strateg...

Atlantis IT group
Toronto, Ontario

Role name: Database Administrator</div> <div>Toronto, ON</div> <div><br /> Role Description: AWS RDS DBA, AWS CDK, Oracle/postgres/DB2, CI CD pipeline, encryption, redshift etc. Competencies: Digital : Amazon Web Service(AWS) Cloud Computing, Digital : Amazon Web Servic...

123Dentist
Toronto, Ontario

We offer professional advancement opportunities, competitive compensation, corporate sponsored incentive programs such as phantom stock option plan for full-time employees (a. You will leverage data to inform key business decisions and use automation tools to improve and streamline finance and opera...

Chronos Consulting
Canada

The Professional Services Engineer is a key role in the Professional Services (PS). Our client is a leading provider of mobile internet optimization solutions that help telecom operators to improve network efficiency, speed and optimize video traffic. They are looking for a Professional Services Eng...

University of Toronto
Toronto, Ontario

Under the general direction of the Executive Director, Enterprise Applications & Solutions Integration or their designate, you will be part of a dynamic, collaborative and innovative team focused on delivering transformational technology solutions to the University. The Enterprise Applications and S...

Intersog
Toronto, Ontario

A strong foundation in data ingestion, storage, orchestration, and management on cloud platforms, with a preference for Azure or AWS. Hands on experience in building metadata-driven, reusable design patterns for data pipeline, orchestration, ingestion patterns (batch, real time); . A Data Engin...