Senior Data Engineer
Data Engineer with Databricks & Spark Experience
We are looking for a data engineer who has strong experience in building scalable and reliable data pipelines using Databricks and Spark.
You will be working with various data sources and formats, and transforming them into valuable insights for our business.
Responsibilities
- Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed
- Optimize data pipelines for performance, scalability, and reliability
- Ensure data quality and integrity throughout the data lifecycle
- Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs
- Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations
- Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management
- Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community
- Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies
Qualifications
- Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent work experience
- 5+ years of experience in data engineering with Databricks and Spark
- Proficient in SQL and Python, and familiar with Java or Scala
- Experience with cloud platforms, such as Azure or AWS
- Experience with big data technologies, such as Kafka, Hadoop, Hive, etc.
- Experience with data warehouse and data lake concepts and architectures
- Experience with data integration and ETL tools, such as Azure Data Factory or Talend
- Experience with data visualization and reporting tools, such as Power BI or Tableau
- Strong analytical and problem-solving skills
- Excellent communication and teamwork skills
Senior Data Engineer
Strong knowledge of data governance, data quality, and data security best practices. Has led/managed a team of Data analysts / Data engineers in his recent position. Support the definition of the International Wealth Management data hub strategy and roadmap to drive specific operational data capabil...
Senior Data Engineer, Services
As a Senior Data Engineer, you will play an instrumental role in advancing our core machine learning solution. You will work alongside scientists and engineers in a collaborative environment, contributing features and data pipelines to power our core recommender systems and our DaVinci Personalizati...
Senior Data Engineer
Contribute to an orchestration layer of complex data transformations, refining raw data from source into targeted, valuable data assets using data vault methodology for consumption in a governed way. You will identify, design and implement the internal processes to support data-driven decision makin...
Senior Data Engineer - ETL Developer
Incorporate core data management competencies including data governance, data security and data quality. Travelers Data Engineering team constructs pipelines that contextualize and provide easy access to data by the entire enterprise. As a Data Engineer, you will play a key role in growing and trans...
Senior Data Engineer, Mapping
To strengthen our efforts, we are hiring a Senior Data Engineer helping us making data driven decisions. As a Data Engineer at Lyft, you will be tasked with developing robust data infrastructure—encompassing data transport, collection, and storage—and providing services that enable our leadership to...
Data & Analytics - Senior Data Visualization Engineer
Proven skills, education, and/or applicable certifications in: Experience creating custom data visualizations and data experiences using modern tools (Tableau, Looker, Power BI)Ability to lead the design process from ideation to polished design, using sketches, wireframes, UX scenarios and mockups t...
Senior Data Engineer - Capital Market
Role: Data Engineer with Capital Market. ...
Senior Software Engineer, Data Analytics
Experience in an analytics focused role - analytics engineer, data engineer or data scientist with a strong engineering background. You will join a centralized data organization, consisting of data engineers, machine learning engineers, scientists and analysts. Senior Software Engineer, Data Analyti...
Senior Software Engineer, Data Analytics
Our Data Science & Engineering team can already boast an impressive suite of systems, tools, and applications to support and utilize our data. At SoundHound, we view our data as our biggest asset. We aspire to leverage our data to make informed decisions to steer product development, marketing, and ...
Senior Software Engineer-Performance/Data/Java
As a Senior Software Engineer (Performance) at PointClickCare, you will be part of a focused team with the ability to have a significant impact on the performance, scalability, and reliability of our Data Platform. We've amassed the richest senior care dataset making our market density untouchable a...