AWS Data Engineer

AWS Databricks Engineer

Toronto - Hybrid (3 days Tuesday to Thursday Work from Office, 2 days Monday and Friday Remote)

Primary Competency Proficiency Details

Digital : Apache Spark E2||Digital : Kafka E2||Digital : Databricks E2

Responsibilities

• Design, develop, and maintain scalable and robust data pipelines on Databricks.
• Collaborate with data scientists and analysts to understand data requirements and deliver solutions.
• Optimize and troubleshoot existing data pipelines for performance and reliability.
• Ensure data quality and integrity across various data sources.
• Implement data security and compliance best practices.
• Monitor data pipeline performance and conduct necessary maintenance and updates.
• Document data pipeline processes and technical specifications.

Qualifications

Skills