Data Engineer
Title : Data Engineer
Type : Contract
Location : Toronto
Environment : Hybrid
We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will have extensive experience in designing, building, and maintaining scalable data pipelines and platforms using cloud-based technologies, particularly within the Microsoft Azure ecosystem.
The role will involve collaborating closely with cross-functional teams to understand data requirements, architect solutions, and drive the implementation of best practices in data engineering.
Key Responsibilities :
- Design, develop, and deploy end-to-end data pipelines and ETL workflows using Azure Data Factory (ADF), Azure Synapse Analytics, and Databricks.
- Collaborate with data scientists and analysts to understand data requirements and translate them into scalable and efficient data solutions.
- Implement data governance and security best practices to ensure the integrity and confidentiality of sensitive data.
- Automate infrastructure provisioning, configuration management, and deployment processes using Azure DevOps, Ansible, and HashiCorp products (Terraform, Vault, Consul, etc.).
- Optimize data pipelines for performance, reliability, and cost-efficiency, leveraging advanced techniques such as partitioning, caching, and parallel processing.
- Monitor and troubleshoot production data pipelines, ensuring high availability and uptime.
- Stay abreast of emerging technologies and best practices in data engineering, continuously evaluating and recommending improvements to our data infrastructure.
Qualifications :
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in data engineering, with a focus on building and optimizing data pipelines in a cloud environment.
- Strong proficiency in Microsoft Azure cloud services, including Azure Data Factory, Azure Synapse Analytics (formerly SQL Data Warehouse), and Azure Databricks.
- Hands-on experience with infrastructure-as-code (IaC) tools such as Azure DevOps, Ansible, or HashiCorp products (Terraform, Vault, Consul, etc.).
- Solid understanding of data modeling, relational databases, and distributed computing principles.
- Experience with containerization technologies (Docker, Kubernetes) and microservices architecture is a plus.
- Excellent problem-solving skills and ability to thrive in a fast-paced, collaborative environment.
- Strong communication skills and ability to effectively communicate technical concepts to non-technical stakeholders.
Data Engineer
Are you looking for a job as Data Engineer ? Currently we are searching for a Data Engineer in. to work for SearchLabs. Being a Data Engineer for SearchLabs you will be in charge of..
Data Engineer
The role. In collaboration with the Business Intelligence (BI) Manager, the Data Engineer will undertake.. This includes the design and implementation of the Data Lake, as well as the optimization of data..
Data Engineer
The role. In collaboration with the Business Intelligence (BI) Manager, the Data Engineer will undertake.. This includes the design and implementation of the Data Lake, as well as the optimization of data..
Data Engineer
Data Engineer. Reports to. Senior Manager Decision Support and Data Insights. Job Summary. We are.. This role will collaborate closely with our data scientist to enhance development practices, set up unit..
Data Engineer
What You Will Do Work and develop in the Data Science and Data Engineering area Skills and Experience You Will Need Required 2 5 years of experience with the following..
data engineer
Education. Secondary (high) school graduation certificate Experience. 2 years to less than 3 years Work setting Office building Tasks Collect and document user's requirements Lead ..