Data Engineer - DataBricks
Job Title : Data Engineer
Location : Anywhere in Canada (100% remote)
Duration : 12+ months
Position Overview :
We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, with expertise in DataBricks and Data Lake technologies.
This role will involve designing, building, and optimizing data pipelines and analytical solutions to support our organization's data-driven initiatives.
Key Responsibilities :
Design, develop, and maintain scalable data pipelines using DataBricks for data ingestion, transformation, and processing.
Implement and optimize DataBricks jobs to handle large volumes of data efficiently.
Collaborate with data architects and analysts to design and implement data lake solutions that meet business requirements.
Develop and maintain data lake infrastructure on platforms such as AWS S3, Azure Data Lake Storage, or Google Cloud Storage.
Write complex SQL and Spark queries for data analysis, aggregation, and reporting within the DataBricks environment.
Implement data governance, security, and access control policies to ensure data quality and compliance.
Monitor and troubleshoot data pipeline issues, identify performance bottlenecks, and implement optimizations.
Stay updated with the latest DataBricks features, best practices, and data engineering trends.
Qualifications :
Bachelor's degree in Computer Science, Engineering, or related field.
Proven experience working as a Data Engineer or similar role.
Strong proficiency in DataBricks and Apache Spark for big data processing.
Experience working with Data Lake architectures and technologies.
Solid understanding of cloud platforms such as AWS, Azure, or Google Cloud Platform.
Proficiency in SQL for data querying and manipulation.
Experience with data modeling, ETL processes, and data warehousing concepts.
Strong problem-solving and analytical skills.
Effective communication and collaboration abilities.
Preferred Qualifications :
Master's degree in Computer Science, Engineering, or related field.
Certifications in DataBricks or relevant cloud platforms.
Experience with other big data technologies such as Hadoop, Kafka, or Apache Flink.
Familiarity with containerization technologies such as Docker and Kubernetes.
Data Engineer - DataBricks
Job Title. Data Engineer Location. Anywhere in Canada (100. remote) Duration. 12. months Position.. Key Responsibilities. Design, develop, and maintain scalable data pipelines using DataBricks for data..
Data Engineer (Databricks)
SQL Server, Neo4J, Elastic, Cosmos). Experience building and optimizing data pipelines on Azure Databricks. In depth knowledge and hands on experience with Data Engineering, Data Warehousing and..
Senior Data Engineer(Databricks) - Toronto
We are looking for a Machine Learning Engineer who has strong experience in building systems that.. Leverage appropriate storage hardware and data formats to improve read re read efficiency. Identify and..
Senior Data Engineer
What We're Looking For. QA Engineer. We are looking for an experienced Quality Assurance (QA) Engineer.. Cucumber) Expertise on test data management SQL or other database experience JSON experience 5. years of..
Platform Engineer (Databricks)
Overview of the role. Guidepoint seeks an experienced Data Engineer as an integral member of the Toronto.. Experience building and optimizing data pipelines on Azure Databricks. In depth knowledge and hands on..
Azure Databricks
Strong experience in any one of the programming languages, like Python. PySpark. Hands on experience with Azure Data Factory and its eco system, Databricks, CI CD (DevOps), Clinical data. Should..
Devops Engineer (Databricks/Snowflake)
Job Opportunity. Senior Data Engineer Our client, a prominent player in the high tech e training.. Senior Data Engineer Industry. High tech e training Location. Remote Tech Stack. Databricks PySpark, SQL..
Senior Data Engineer (Azure Databricks/Java)
Title. Sr. Azure Data Engineering Specialist (Databricks and Synapse) Location. This role can be.. Expertise in Azure Databricks and Synapse experience is mandatory for this role. This is a hands on role..
Databricks Instructor
Tune Databricks for performance and utilization optimization.Provide infrastructure guidance of.. Powered by innovative technology, real time data, and hard to source expertise, we help our clients to..
Platform Engineer (Databricks)
SQL Server, Neo4J, Elastic, Cosmos).Experience building and optimizing data pipelines on Azure Databricks.In depth knowledge and hands on experience with Data Engineering, Data Warehousing and..