Talent.com
VDart Inc
PySpark Python DeveloperVDart Inc • Mississauga, ON, Canada
PySpark Python Developer

PySpark Python Developer

VDart Inc • Mississauga, ON, Canada
6 hours ago
Job type
  • Full-time
  • Quick Apply
Job description

Job Title: PySpark Python Developer

Location: Mississauga, ON

Duration: Full Time


Required Qualifications:

  • At least 4 years of Information Technology experience
  • 4+ years of experience in Big Data technologies.

Strong expertise in:

  • Apache Spark (Core, SQL, DataFrames, RDDs)
  • Scala programming
  • PySpark
  • Hands-on experience with:
  • Kafka (real-time streaming)
  • Hadoop ecosystem (HDFS, Hive, Impala)
  • NoSQL Databases (HBase, MongoDB, Couchbase)
  • Strong understanding of distributed computing concepts and data processing frameworks.
  • Experience in building ETL/data pipelines for large-scale datasets.
  • Proficiency in SQL and data modeling.

Preferred Qualifications:

  • Hands-on experience with data lakes, data warehouses, and scalable ETL pipeline design, including batch and real-time processing architecture.
  • Strong understanding and practical exposure to Agile software development methodologies (Scrum) and SDLC practices.
  • Proven experience in Banking domain, supporting use cases such as fraud detection, risk analytics, regulatory reporting, and customer insights.
  • Excellent analytical, problem-solving, and communication skills, with the ability to translate business requirements into scalable technical solutions.
  • Demonstrated ability to work effectively in cross-functional, multi-stakeholder environments, collaborating with Business, Data Engineering, and Architecture teams.
  • Experience with real-time data streaming frameworks such as Kafka and Spark Streaming for low-latency processing.
  • Understanding data modeling concepts (dimensional modeling, snowflake schemas) to support analytics workloads.
  • Experience and desire to work in a global delivery environment.

Key Responsibilities:

  • Design and develop large-scale data processing pipelines using Apache Spark (Scala & PySpark)
  • Build and optimize batch and real-time data processing workflows using Spark, Kafka, and Hadoop ecosystem
  • Develop Spark applications using RDDs, DataFrames, and Spark SQL for complex transformations
  • Develop and optimize PySpark applications leveraging joins, Spark DAG execution flow, stage optimization, transformation techniques, and streaming with dynamic allocation and failover handling.
  • Implement streaming pipelines using Kafka and Spark Streaming / Structured Streaming
  • Develop and maintain HDFS, Hive, NoSql and Impala-based data lake solutions
  • Convert existing SQL/Hive workloads into optimized Spark jobs for improved performance
  • Work with ETL pipelines to ingest, cleanse, transform, and process large datasets
  • Optimize performance through partitioning, caching, serialization, and tuning techniques
  • Handle data formats such as Parquet, ORC, Avro, JSON
  • Integrate multiple data sources including streaming systems, flat files RDBMS, and APIs
  • Collaborate with cross-functional teams to understand business requirements and translate them into scalable technical solutions
  • Ensure data quality, reliability, and performance monitoring across pipelines
  • Participate in code reviews, design discussions, and best practices implementation

Key Skills:

  • Distributed Data Processing.
  • Spark Optimization & Performance Tuning.
  • Real-time Data Streaming.
  • Data Modeling & ETL Design.
  • Problem-solving and Analytical Thinking.
  • Strong Communication & Stakeholder Management.

Nice to Have:

  • Exposure to Machine Learning pipelines or MLOps workflows.
  • Experience with Databricks platform.
  • Experience with AWS/GCP.
Create a job alert for this search

PySpark Python Developer • Mississauga, ON, Canada

Similar jobs

Distributed Systems Software Engineer, Python / Go

Canonicalmississauga, peel region, Canada
Full-time

Distributed Systems Software Engineer, Python / Go.Distributed Systems Software Engineer, Python / Go.Distributed Systems Software Engineer, Python / Go.Be among the first 25 applicants.Distributed... Show more

 • Promoted

Remote Lead Data Engineer - Databricks & PySpark

LumenaltaMississauga, Peel Region, CA
Remote
Full-time

A technology company is seeking an experienced Data Engineering Lead to guide technical direction for critical initiatives in data engineering.This fully remote position requires 10+ years of exper... Show more

 • Promoted

Power BI Data Engineer — Azure Data & Dashboards Lead

Lakefield Veterinary GroupBurlington, Halton Region, CA
Full-time

A veterinary services provider seeks a Data Analyst responsible for collecting and analyzing data to support strategic initiatives.The role involves creating PowerBI dashboards, working closely wit... Show more

 • Promoted

Algebra Private Tutoring Jobs Bond Head

SuperprofBond Head, Ontario (ON), CA
Full-time

Superprof is Canada's tutoring platform, and we're actively recruiting passionate tutors! Whether you're a student, a professional, or simply someone who loves teaching, join the largest community ... Show more

 • Promoted

SAP Datasphere Developer

Insight GlobalMississauga, Peel Region, CA
Temporary

Get AI-powered advice on this job and more exclusive features.This range is provided by Insight Global.Your actual pay will be based on your skills and experience — talk with your recruiter to lear... Show more

 • Promoted

Data Engineer | Python & Spark Specialist

AltimetrikMississauga
Full-time

A technology firm in Peel Region, Mississauga is seeking a Mid-Senior level software developer proficient in Python and Spark.This full-time role involves designing, developing, and maintaining sof... Show more

 • Promoted

Full Stack Developer

Measure Marketing Results Inc.Oakville, ON, CA
Full-time

Full Stack Developer – Measure Marketing Results Inc.Join the team as a WordPress Full Stack Web Developer and AI Tools Specialist.Your role will involve building high-performance, fully customized... Show more

 • Promoted

Software Engineer, Python

AutoSyncMississauga, Peel Region, CA
Full-time

TRADER Corporation is a trusted Canadian leader in online media, dealer and lender services.AutoSync, and Dealertrack Canada.Quebec) offers the largest inventory of new and used cars in Canada, wit... Show more

 • Promoted

Senior Azure Cloud Engineer — API & Integration Lead

BDO CanadaOakville
Full-time

A leading Canadian consulting firm in Ontario is seeking a Senior Developer to design and maintain cloud-based solutions on Microsoft Azure.The role requires collaboration with cross-functional tea... Show more

 • Promoted

Senior SharePoint & Power Automate Contractor | M365 Migrations

isgSearchVaughan
Full-time

A leading real estate organization based in North Toronto is seeking a skilled IT professional to support its large-scale business transformation initiatives.The ideal candidate will have extensive... Show more

 • Promoted

Integration Developer: Java and Agile Skills

JLI Consulting Talent SearchVaughan
Full-time

Join an innovative Agile team as an Integration Developer specializing in Java.Your expertise in RESTful APIs and framework integration will drive impactful IT solutions in a collaborative environm... Show more

 • Promoted

Remote Python Developer - Cloud Automation & Geospatial

CATALYST MicroservicesMississauga, Peel Region, CA
Remote
Full-time

A technology solutions company is seeking a Python Developer to design and maintain cloud deployment pipelines and automation workflows.With an emphasis on sustainability, this remote role involves... Show more

 • Promoted

Lead BI & Reporting Developer

Husky Injection Molding Systems Ltd.Bolton, ON, CA
Full-time

An innovative company in Bolton is seeking a Business Intelligence & Reporting Developer to enhance its data-driven decision-making capabilities.This role offers the chance to lead the design and s... Show more

 • Promoted

Data Warehouse Support Engineer - SQL, Python & AI

CAA Club GroupVaughan
Full-time

A leading member-based organization is looking for a candidate to assist with troubleshooting application system issues and lead various projects.Candidates must have at least 1 year of experience ... Show more

 • Promoted

Backend Python Engineer – Scalable, AI-Driven AWS Systems

Scribd, Inc.Mississauga, Peel Region, CA
Full-time

A global digital library service is seeking a Software Engineer II to design and optimize scalable systems within their ML Data Engineering team.The role focuses on building event-driven solutions ... Show more

 • Promoted

Senior Data Engineer - Python, PySpark and OpenShift Lead

Compunnel, Inc.Mississauga
Full-time

A leading data engineering firm is seeking a Data Engineer to design and maintain scalable data pipelines using Python and PySpark.This role requires managing data storage solutions, orchestrating ... Show more

 • Promoted

Senior ETL Developer - Hybrid Data Pipeline Lead

Randstad DigitalOakville
Full-time

A leading insurance firm in Canada is seeking a Senior ETL Developer to join their hybrid team.The role involves building efficient data pipelines and operationalizing high-quality data solutions.C... Show more

 • Promoted

CRM and Data Management Advisor - Spanish Required

Cuso InternationalOakville, Ontario
Permanent

This Volunteer Placement is Located in:.Please submit a Spanish Resume and Statement of Interest.Open to Canadian Citizens and Permanent Residents of Canada only.This placement is an opportunity to... Show more

 • Promoted

Software Development Lead Spiria

SpiriaOakville
Full-time

Shape software solutions as Spiria's Software Development Lead.This position focuses on technical leadership, mentoring, and innovation in creating client-oriented applications.As the Software Deve... Show more

 • Promoted

Real-time Spark/Scala Big Data Engineer (Contract)

KTek ResourcingMississauga, Peel Region, CA
Full-time

An established industry player is seeking a skilled mid-senior level professional to join their team in Information Technology.This role emphasizes expertise in Spark, Scala, and Hadoop, with a foc... Show more