Data Engineering Lead and Data EngineersAlltech Consulting Services • Ottawa
Data Engineering Lead and Data Engineers
Alltech Consulting Services • Ottawa
30+ days ago
Job type
- Full-time
Job descriptionBuild and optimize data pipelines for efficient data ingestion, transformation and loading from various sources while ensuring data quality and integrity. Design, develop, and deploy Spark program in databricks environment to process and analyze large volumes of data. Experience of Delta Lake, DWH, Data Integration, Cloud, Design and Data Modelling. Proficient in developing programs in Python and SQL Experience with Data warehouse Dimensional data modeling. Working with event based / streaming technologies to ingest and process data. Working with structured, semi structured and unstructured data. Optimize Databricks jobs for performance and scalability to handle big data workloads. Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks. Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions. Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process. Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards. 5+ years Python coding experience. 5+ years – SQL Server based development of large datasets 5+ years with Experience with developing and deploying ETL pipelines using Databricks Pyspark. Experience in any cloud data warehouse like Synapse, Big Query, Redshift, Snowflake. Experience in Data warehousing – OLTP, OLAP, Dimensions, Facts, and Data modeling. Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills. Experience with Cloud based data architectures, messaging, and analytics. Cloud certification(s). Strong experience in Code Optimizations and Performance tuning
Job Description
Must Have
- Snowflake
- Databricks
- SQL
- Python
- ADF
- PySpark
- Data Warehouse Concepts
Job Description :
This position is for a Cloud Data engineer with a background in Python, Pyspark, SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable working with business users along with business analyst expertise in understanding Banking and Capital markets , especially in investment banking would be a big plus. Strong Database fundamentals is a must along with PySpark Coding expertise.
Major Responsibilities :
Skills :
Create a job alert for this search
Data Engineering Lead and Data Engineers • Ottawa
Similar jobs
A global survey platform is seeking an experienced Data Scientist to lead projects and mentor junior team members.You will advocate for data-driven practices while collaborating across teams.Requir...Show more
Data Engineer with AWS, Glue, Lambda, SQL, Python, Redshift.Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred).Must be able to...Show more
A leading survey platform is seeking a senior data science leader to guide and mentor junior team members while driving machine learning projects.
Candidates should have over 12 years of data scienc...Show more
A leading global consulting firm in Ottawa seeks a Manager - Solution Architect / Tech Lead to enhance client capabilities in data analytics.
This role involves supporting key projects, building inter...Show more
Connect with us to discover our latest job opportunities! Even if nothing suits you right now, stay in touch — your perfect role may be just around the corner! .
Noda is a data and analytics co...Show more
Our client, a leading organization in the public sector, is seeking a.As a SAP SuccessFactors Data Lead, you will be part of the HR and Data Management department supporting SAP HR modules.The idea...Show more
A leading analytics consulting firm in Ontario is seeking a skilled Data Engineer to architect and implement advanced analytics capabilities.
You will work with large data sets and visualization too...Show more Data Engineer with AWS, Glue, Lambda, SQL, Python, Redshift.Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred).Must be able to...Show more 
We are hiring a Data Lead to support a large-scale enterprise transformation program delivering a modern Knowledge Management (KM) solution.
In this role, you will lead data-related activities that ...Show more
We Are • •Accenture Technology – is an exciting place to be! Accenture helps companies rotate to New IT with services that drive continuous quality and efficiency and shape their digital future.We of...Show more
A public service organization is seeking an experienced Database Architect for a future opportunity.This role focuses on leading assessments and modernizing the enterprise data ecosystem.Candidates...Show more
A leading technology firm in Ottawa is seeking a Software Engineering Manager to drive the next generation of digital forensics tools.
You will lead technical teams, establish modern AI practices in...Show more
As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.
Headquartered in South Florida with a remote-...Show more
The Senior Data Engineer Service Provider will be responsible for leading the delivery of data on data and analytics platform.
These services are at the forefront of building out data engineering pr...Show more
A leading semiconductor firm located in Ottawa is searching for a Senior ASIC Design Engineer.The role requires a minimum of 8 years in ASIC design with strong skills in RTL logic and lab troublesh...Show more
A leading cybersecurity firm is seeking a Director of Engineering to lead its Email Archive portfolio, focusing on compliance and security.
The role involves guiding distributed teams, enhancing pro...Show more
Senior Databricks Data Engineer.Location : In Toronto, Montreal, or Ottawa.Job Type : Full-time or Contract.Rate and salary : up to $100 per hour (contract), or up to $160,000 per year (employee).Seni...Show more
TekRek has partnered with a fast growing firm specializing in AI enablement and data optimization.Our client works with some of the top tier technology firms in Silcom Valley.With rapid growth and ...Show more
Senior ML Engineer & Tech Lead — NLP, MLOps, Mentoring
SurveyMonkey • Ottawa H2B, ON, Canada
Remote
Full-time
Last updated: 24 days ago • Promoted
AWS data engineer DWIDC5725169
Compunnel Inc. • ottawa, on, ca
Full-time
Last updated: 1 day ago • Promoted
Senior Staff ML Engineer & Data Leadership (Hybrid)
Tree Trust • Ottawa
Full-time
Last updated: 2 days ago • Promoted
AI & Data Leader : Solution Architect & Tech Lead
EY • Ottawa
Full-time
Last updated: 17 days ago • Promoted
Data Engineer
Noda • Ottawa, Ontario, CA
Full-time
Quick Apply
Last updated: 3 days ago
SAP SuccessFactors Data Lead
Experis • Ottawa, ON, Canada
Full-time
Last updated: 7 days ago • Promoted
Data Engineer : Snowflake DW & ELT Visionary
Tiger Analytics • Ottawa
Full-time
Last updated: 17 days ago • Promoted
AWS data engineer DWIDC5725169 - Compunnel Inc.
Compunnel Inc. • ottawa, on, ca
Full-time
Last updated: 1 day ago • Promoted
Data Lead
CGI • Ottawa
Full-time
Last updated: 5 days ago • Promoted
Data Centre Engineer
Queer Tech • Ottawa
Full-time
Last updated: 5 days ago • Promoted
Remote Data Architect - Modernize Enterprise Data
Orangutech • Ottawa H2B, ON, Canada
Remote
Part-time
Last updated: 30+ days ago • Promoted
Hybrid Software Engineering Manager — Lead AI‑Driven SaaS
Magnet Forensics • Ottawa H2B, ON, Canada
Remote
Full-time
Last updated: 22 days ago • Promoted
Lead Data Engineer
Launch Potato • Ottawa
Full-time
Last updated: 17 days ago • Promoted
Senior Data Engineer
freelance.ca • Ottawa, Canada
Full-time
Last updated: 24 days ago • Promoted
Senior ASIC Design Engineer for AI / HPC SoCs
Btadesignservices • Ottawa H2B, ON, Canada
Full-time
Last updated: 27 days ago • Promoted
Director of Engineering – Global Data & Compliance Platform
Barracuda Networks Inc. • Ottawa, ON, Canada
Full-time
Last updated: 22 days ago • Promoted
Senior Databricks Data Engineer
KData AI • Toronto, Ottawa, Montreal (Administrative Region)
Full-time
Last updated: 8 hours ago • Promoted • New!
Senior Analytics Engineer
TekRek • gatineau, qc, ca
Full-time
Last updated: 29 days ago • Promoted