Big data engineer • toronto on
Role: AWS Big Data Platform Engineer (Netezza to AWS Migration).Hybrid: 2 days a week in office.We are seeking a highly skilled Senior Platform Engineer with extensive experience in Big Data techno... Show more
Design, deploy and maintain a centralized Azure ADLS Gen2/Databricks/Synapse (or Snowflake) data lake that serves as the single source of truth.Create batch & real‑time ETL/ELT flows (Azure Eve... Show more
Our client, a leading Canadian commercial property and casualty insurance provider, is seeking an experienced Data Quality Engineer (Contract) with 7+ years of hands-on experience in Data Quality, ... Show more
In addition to leading core data capabilities, this role will act as a.This leader will define and execute the.This will give you the following.Partner with business, analytics, risk, and technolog... Show more
About PureFacts Financial Solutions.PureFacts is the leader in the Revenue Performance Management category for wealth and asset management firms.The PureRevenue™ Platform helps organizations maximi... Show more
Citylitics delivers predictive intelligence on where, how, and why infrastructure investments will occur across North America.Infrastructure includes the foundational systems that shape our quality... Show more
We are seeking a Data Engineer to build and maintain scalable data pipelines supporting enterprise analytics and reporting.The role will focus on developing ETL/ELT pipelines using Informatica and ... Show more
Tiger Analytics is a fast-growing advanced analytics consulting firm.Our consultants bring deep expertise in Data Science, Machine Learning and AI.We are the trusted analytics partner for several F... Show more
Financeit is a point-of-sale financing provider serving some of the largest home improvement and retail organizations in Canada.Our platform helps businesses close more sales by offering customers ... Show more
Headquartered in San Francisco, our investors include.Network Engineer - Data for Autonomous Systems annotation.Review real-world data from deployed networks, including logs, configs, telemetry, an... Show more
Financeit is a point-of-sale financing provider serving some of the largest home improvement and retail organizations in Canada.Our platform helps businesses close more sales by offering customers ... Show more
We are looking for a Sr Data Engineer with in-depth expertise in AWS, Databricks, and modern data architecture and data modeling to help build the next generation of our data foundation , including... Show more
Please review the job description below and contact me ASAP if you are interested.Key Requirements and Technology Experience:.Key Skills:SENIOR SPARK DEVELOPER, SCALA BIG DATA DEVELOPER, DATA ENGIN... Show more
BMO is an organization driven by a shared Purpose: Boldly Grow the Good in business and life.It calls on members of its team, to create lasting, positive change for its customers, its communities, ... Show more
Rockstar is recruiting for a fast-growing, mission-driven technology company focused on workforce development.The client is dedicated to building innovative digital solutions that empower individua... Show more
A leading fin-tech company is hiring a Principal Data Engineer for a full-time opportunity in Toronto, ON (Hybrid - 2-3 days onsite).This role is ideal for a senior engineer who has strong experien... Show more
Hybrid (2 days onsite / 3 days remote) - Toronto.We have a great new opportunity to support one of our Loyalty Sector clients in a contract capacity!.Please see below for more information on the po... Show more
What We do: iQmetrix is a global provider of Interconnected Commerce software solutions for telecom retail.Interconnected Commerce is an AI-native telecom commerce platform that acts as a system of... Show more
Join Livingston and grow your career in the constantly changing world of international trade.Livingston is a market leader offering customs brokerage, international trade consulting, compliance and... Show more
Champions a customer-focused culture to deepen client relationships and leverage broader Bank relationships, systems and knowledge.Designs, builds, and maintains scalable data architectures and dat... Show more
Role: AWS Big Data Platform Engineer (Netezza to AWS Migration)
Hybrid: 2 days a week in office
Job Summary: We are seeking a highly skilled Senior Platform Engineer with extensive experience in Big Data technologies, AWS Cloud, and data platform engineering. The ideal candidate will have strong expertise in designing, developing, and supporting scalable data pipelines and distributed systems, along with hands-on experience in Big Data ecosystem tools, AWS services, and real-time data processing. This role involves working on data platform modernization, cloud migrations, Data Warehousing and ETL in a fast-paced enterprise environment.
Key Responsibilities
· Design, develop, maintain and support scalable data pipelines using Big Data and AWS technologies.
· Lead and support data platform migration initiatives (On-Prem to AWS Cloud), ideally with Netezza background.
· Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar.
· Implement and manage AWS services such as EMR, S3, EC2, Redshift, Glue, and Airflow.
· Build and optimize data workflows and orchestration pipelines using Airflow.
· Work with real-time streaming technologies such as Kafka and Spark Streaming.
· Perform data ingestion, transformation, and validation from multiple data sources.
· Optimize SQL queries for performance and scalability.
· Monitor system performance, troubleshoot issues, and ensure system reliability.
· Collaborate with cross-functional teams including developers, architects, and infrastructure teams.
Required Technical Skills
· Cloud Technologies
· Strong experience in AWS Cloud services:
· EMR, EC2, S3, VPC, RDS, Redshift
· AWS Glue, IAM, CloudWatch, CloudFormation
· Airflow (or AWS Managed Workflows)
· Databases
Experience working with:
· Netezza is mandatory
· SQL-based systems: SQL Server and PostgreSQL
· Data warehouses: Teradata, Redshift, Netezza
· ETL Tools
Hands-on experience with:
· SSIS
· Programming & Scripting
Strong proficiency in:
· SQL
· Shell scripting
Good to have:
· Python
· Operating Systems
· Strong experience in Unix/Linux environments
Key Qualifications
· 10+ years of experience in Data Engineering / Big Data / Platform Engineering.
· Hands-on experience in AWS and Hadoop ecosystems.
· Experience in data migration projects (e.g., Hadoop to AWS, DB migrations).
· Strong understanding of data warehousing concepts and ETL design patterns.
· Expertise in query optimization and performance tuning.
· Experience supporting production data platforms.
· Bachelor’s degree in Computer Science, Engineering, or related field.
Preferred Qualifications
· Experience in Banking / Financial Services domain
· Experience with data lake architecture and governance
· Knowledge of DevOps tools and CI/CD pipelines
· Experience in platform engineering and framework development
Soft Skills
· Excellent problem-solving and analytical skills
· Strong communication and collaboration abilities
· Ability to work independently and lead initiatives
· Experience mentoring team members
Nice-to-Have
· Experience with tools like Ansible, Git, JIRA
· Exposure to NoSQL databases (HBase, MongoDB)
· Experience in data masking and security frame
