RQ08100 - Software Developer - ETL - Senior

freelance.caToronto, Canada

30+ days ago

Job description

Job Opportunity : Senior Software Developer - ETL (RQ08100)

Duration : Nov 18, 2024 – Mar 31, 2025 (Potential extension)

Hours : 7.25 hours / day

Application Requirements (Submit by Nov 13, 12 : 30 PM EST) :

Updated Resume (Word format)
Candidate Summary & References
Expected hourly rate
Visa Status
LinkedIn Profile

Apply via email : [email protected]

For daily updates , join the WhatsApp group.

Must-Have Skills :

7+ years of experience with ETL tools (Microsoft SSIS, T-SQL, Oracle)

2+ years of experience with Azure Databricks, Delta Lake, and Python / PySpark

Proficiency in Medallion Architecture (Bronze, Silver, Gold) and Delta Lake

Experience with Change Data Capture (CDC) using tools like Oracle GoldenGate

Strong understanding of Databricks Workflows for task scheduling and orchestration

Hands-on with SQL Server, data modeling, and ETL pipeline optimization

Role Overview :

Key Focus : Design, develop, optimize, and maintain ETL processes in Databricks for data lakes and analytics.

Core Responsibilities : Develop pipelines for data transformation using Azure Databricks and PySpark . Implement CDC for real-time data ingestion and maintain data integrity. Optimize data pipelines using Delta Live Tables and Medallion Architecture . Perform data modeling, schema design, and data quality checks. Collaborate with data architects, analysts, and business stakeholders.

Deliverables :

ETL Development : Create pipelines for data transformation and ingestion.

Data Optimization : Review and enhance ETL performance.

Go-Live Support : Manage deployment and troubleshoot post-launch issues.

Documentation & Knowledge Transfer : Develop comprehensive documentation and transfer knowledge to Ministry staff.

Evaluation Criteria :

Development & ETL Experience (60 points) Deep experience in ETL pipelines, Delta Lake, and Databricks.

Design Documentation & Analysis (20 points) Strong skills in documentation, error handling, and troubleshooting.

Certifications (10 points) Relevant certifications like Azure Data Engineer or Databricks Certified Data Engineer.

Communication & Leadership (10 points) Strong collaboration skills and experience in Agile environments.

Total : 100 points

Notes :

Hybrid work arrangement : 3 days onsite at 5775 Yonge Street, Toronto.

Remote work may be allowed initially but requires onsite presence as per Ministry directives.

Public sector experience is an asset.