Talent.com
LONG FINCH TECHNOLOGIES
Data Engineer (PYSPARK)LONG FINCH TECHNOLOGIES • Mississauga, Ontario, Canada
Data Engineer (PYSPARK)

Data Engineer (PYSPARK)

LONG FINCH TECHNOLOGIES • Mississauga, Ontario, Canada
23 days ago
Job type
  • Full-time
Job description

Role: Python & PySpark Data Engineer

Overview

We are looking for a Data Engineer with strong expertise in Python and PySpark to design, build, and optimize scalable data pipelines. You will work with large datasets, distributed systems, and cloud platforms to enable data-driven decision-making.

Key Responsibilities

1. Data Pipeline Development

  • Design and build ETL/ELT pipelines using Python and PySpark
  • Process large-scale structured and unstructured data
  • Ensure high performance and reliability of data workflows

2. Big Data Processing

  • Use Apache Spark (especially PySpark) for distributed data processing
  • Optimize Spark jobs (partitioning, caching, joins, etc.)
  • Handle batch and near real-time data processing

3. Data Integration

  • Ingest data from multiple sources: APIs, databases, flat files, streaming systems
  • Work with tools like Apache Kafka for real-time pipelines
    • Ensure data consistency and integrity

    4. Data Modeling & Storage

    • Design scalable data models (star/snowflake schemas)
    • Work with:
      • Data lakes (e.g., Amazon S3)
      • Data warehouses (e.g., Snowflake, Amazon Redshift)

    5. Performance Optimization

    • Tune SQL queries and Spark jobs
    • Optimize memory usage and job execution time
    • Implement efficient partitioning and indexing strategies

    6. Cloud & DevOps

    • Work on cloud platforms like:
      • Amazon Web Services
      • Microsoft Azure
      • Google Cloud Platform
    • Build CI/CD pipelines for data workflows
    • Use containerization tools like Docker

    7. Data Quality & Governance

    • Implement validation checks and monitoring
    • Ensure data accuracy, lineage, and governance
    • Work with logging and alerting systems

    Required Skills

    Core Technical Skills

    • Strong programming in Python
    • Expertise in PySpark / Apache Spark
    • Advanced SQL knowledge
    • Experience with distributed computing

    Big Data & Tools

    • Hands-on with:
      • Hadoop ecosystem
      • Apache Hive
      • Apache Airflow

    Data Engineering Concepts

    • ETL/ELT design
    • Data warehousing & modeling
    • Batch vs streaming architectures

    Cloud & Storage

    • Experience with cloud data services (S3, BigQuery, ADLS, etc.)
    • Understanding of data lake architecture

    Preferred / Nice-to-Have Skills

    • Real-time processing (Kafka, Spark Streaming)
    • Knowledge of Delta Lake or Apache Iceberg
    • Experience with Databricks
    • Basic understanding of machine learning pipelines
    • Familiarity with DevOps tools (CI/CD, Terraform)

Create a job alert for this search

Data Engineer (PYSPARK) • Mississauga, Ontario, Canada

Similar jobs

Data Engineer

KUBRAMississauga, Peel Region, CA
Full-time

We’re looking for a Data Engineer to join our Product Development - Business Intelligence team.In this role, you’ll build and maintain data pipelines and infrastructure that power BI products and c... Show more

 • Promoted

Data Engineer - Minimum 12+ years experience - oakville

NLB Servicesoakville, on, ca
Full-time

Lead the migration from Glue/Redshift to Databricks.Deep, hands-on experience with Apache Spark (PySpark and Spark SQL), including performance tuning (partitioning, shuffles, skew, caching, file si... Show more

 • Promoted

Remote Lead Data Engineer - Databricks & PySpark

LumenaltaMississauga, Peel Region, CA
Remote
Full-time

A technology company is seeking an experienced Data Engineering Lead to guide technical direction for critical initiatives in data engineering.This fully remote position requires 10+ years of exper... Show more

 • Promoted

Azure Databricks Engineer

Tredence Inc.oakville, on, ca
Full-time

We are seeking a skilled Azure Databricks Engineer with strong experience in Azure Data Factory (ADF) to design, develop, and maintain scalable data pipelines and analytics solutions on the Azure c... Show more

 • Promoted

Data Engineer specializing in PySpark

SynechronMississauga, Peel Region, CA
Full-time

Elevate your career at Synechron as a Data Engineer specializing in PySpark.Leverage your experience in data modeling and release management to support banking analytics.Join Synechron's dynamic te... Show more

 • Promoted

Lead Data Engineer

McKesson’s CorporateMississauga, Peel Region, CA
Full-time

McKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare.We are known for delivering insights, products, and services that make quality care more accessibl... Show more

 • Promoted

Data Engineer | Python & Spark Specialist

AltimetrikMississauga
Full-time

A technology firm in Peel Region, Mississauga is seeking a Mid-Senior level software developer proficient in Python and Spark.This full-time role involves designing, developing, and maintaining sof... Show more

 • Promoted

Lead Data Engineer - Python, PySpark & SQL

Princeton IT Services, IncMississauga, Peel Region, CA
Full-time

Lead Data Engineer – Python, PySpark & SQL.Build scalable data ingestion and transformation pipelines using Python, PySpark, and SQL.Process raw CSV/text files from AWS S3, including validating hea... Show more

 • Promoted

Data Engineer - Google Cloud Expertise

DatatonicMississauga, Peel Region, CA
Full-time

Join Datatonic as a Senior Data Engineer and leverage your expertise in Google Cloud technologies.Focus on driving data transformation across diverse projects in an all-remote environment.As part o... Show more

 • Promoted

Data Engineer Role in Alberta’s Digital Division

PathlionMississauga, Peel Region, CA
Full-time

Be part of Alberta's digital transformation as a Data Engineer.Contribute to innovative projects by creating data pipelines and deriving insights that meet the needs of citizens.As a core member of... Show more

 • Promoted

Lead Data Engineer Innovating Healthcare Data Architecture

Synthesis HealthMississauga, Peel Region, CA
Full-time

Drive data engineering excellence as a Staff Data Engineer and help transform healthcare data practices.Spearhead architecture optimizations and educate teams in a fully remote role.As a leader in ... Show more

 • Promoted

Senior Data Engineer - Data Governance & Platform Lead

XsollaMississauga, Peel Region, CA
Full-time

A global commerce company in Canada is seeking a Data Engineer to lead efforts in building and optimizing the User Platform and Recommendation Engine.The role involves ensuring data accuracy and pe... Show more

 • Promoted

Data Engineer

Charger Logistics Inc.Brampton, Peel Region, CA
Full-time

With over 20 years of experience providing the best logistics solutions, Charger logistics has transformed into a world-class transport provider and continue to grow.Charger logistics invests time ... Show more

 • Promoted

Senior Data Engineer — Spark, Airflow, AWS | Remote QC

Go-ToMississauga, Peel Region, CA
Remote
Full-time

A technology company in Quebec is seeking a Senior Software Developer.You will be responsible for diagnosing data processing issues, ensuring software releases, and preventing data errors.Key quali... Show more

 • Promoted

Data Engineer

Enterprise Solutions Inc.Mississauga, ON, Canada
Full-time

Work Location : Mississauga-Ontario-CA- Onsite from day 1.Salary: CAD$79 k to CAD$90k/annum.Mandatory Skills : Apache Spark, Scala, Big Data Hadoop Ecosystem, Python, SparkSQL.Seeking a Specialist ... Show more

 • Promoted

Data & Analytics Engineer

Amaris Consultingoakville, on, ca
Full-time

Senior Data & Analytics Engineer.This role is ideal for a highly technical professional who can combine hands-on data engineering expertise with platform structuring, infrastructure optimization, a... Show more

 • Promoted

Data Engineer

BorrowellMississauga, Peel Region, CA
Full-time

At Borrowell, we're on a mission to help Canadians feel confident about their money.We empower individuals to take control of their financial futures by providing the tools and insights needed to u... Show more

 • Promoted

Senior Data Engineer (Analytics Focus)

TapMangoMississauga, Peel Region, CA
Full-time

We need someone who can turn millions of messy transactions into clean, fast, "aha moment" analytics for thousands of merchants.You'll build the data pipelines that power dashboards, design the war... Show more

 • Promoted

Senior Data Engineer - dbt/Looker for Unified Analytics

The Data Co.Mississauga, Peel Region, CA
Full-time

A data solutions company is seeking a Senior Data Developer to build and maintain the technical backbone of its analytics platform.This role requires deep expertise in dbt and Looker, with responsi... Show more

 • Promoted

Senior Data Engineer (Datastage experience)

RBCMississauga, Peel Region, CA
Full-time

As a Senior Data Engineer in the Insurance Data Engineering Services team within Insurance Technology, you will provide technical application and functional support to the Insurance business.Your f... Show more