Job Description
Role Description :
- Designing and developing data pipelines using Cloud Technologies based on business requirements and technical design.
- Collaborating with Stakeholders, Business Analysts and Data Architects to assist and translate business requirements to technical solutions
- Develop big data and analytic solutions leveraging new or existing technology to advance client’s all lines of business
- Exploratory data analysis by querying and processing data stores and databases and generate reports.
- Design, upgrade and implement new data workflows.
- Write and maintain technical documentation
- Executes updates, patches, and other activities required to maintain and enhance the operations of on-premise or cloud-based environments
Competencies : Data Structures
Experience (Years) : 4-6
Essential Skills :
- Experience working as Data Engineer with focus on big data processing and / or relational databases with good understanding of SQL
- Experienced with working on Structured and Semi-Structured datasets.
- Experience working with Microsoft Azure Data Platform, specifically Azure SQL Database, Azure Data Factory, ADLS storage account, Azure Data Bricks and DevOps Experienced in creating data pipelines & developing complex and optimized queries
- Experienced with Workflow Management Tools : Airflow, Crontab, CA Workload Automation
- Experience with implementing CI / CD pipeline on GitHub, Jenkins / JGP.
- Experience writing complex SQL and NoSQL jobs to analyze data in traditional DBMS (MS-SQL, Oracle) environments
- Experience with integrating to back-end / legacy environments
- Experience integrating business and technology teams
- Experienced in any of the following programming / scripting languages (SQL, Python, Shell, Scala) Excellent organizational and time management skills, strong business presence with ability to multi-task and effectively deal with competing priorities
Desirable Skills :
- Previous work experience in the financial sector (insurance or benefits industry).
- Knowledgeable or at least have the basic concept of Data Visualization in any of the following tools : Tableau, PowerBI, QlikView / QlikSense
Azure Data Platform Engineer
Minimum of 3 years of experience with Azure data services, including Azure Data Factory, Azure Data Lake, and Databricks. Design, develop, and maintain data pipelines using Azure Data Factory, incorporating XML and JSON data formats for data exchange and transformation. Utilize Azure Databricks for ...
AWS Data Engineer
Job Locatio : Toronto, Canada- Day onsite / Hybrid Model).Look for + Years of experience intern).Implementation partner experience .Mandatory Skills (Expert level).AWS CDK (setting up environments with code).Redshift (RDBMS), Athena, Lambda (uses ).Implement CI/CD pipeline using AWS CI/CD stack.Writ...
Senior Process Data Engineer
Partner with the MSAT Data Science and Digital teams to apply re-usable and efficient data engineering solutions for new and legacy facilities, and when developing process analytics solutions •Translate data analytics outcomes to non-scientific audiences, champion data-driven decision making and emp...
Data Engineer
The Data Engineer should have extensive experience working with very large data sets from the insurance industry and knowledge of building programs/mappings that leverage the Azure and Databricks platforms and Data warehouses. Strong preference will be given to Databricks Certified Data Engineer Ass...
data engineer
Research and document data requirements, data collection and administration policy, and data access rules. Lead and co-ordinate teams of data administrators in the development and implementation of data policies, standards and models. Conduct research regarding availability and suitability of data. ...
Senior Data Engineer
Work with tools in the Microsoft Stack; Azure Data Factory, Azure Data Lake, Azure SOL Databases, Azure Data Warehouse, Azure Synapse Analytics Services, Azure Databricks, Microsoft Purview, and Power Bl. Requires experience of over 4 years and up to 6 years in data modeling, data warehouse design a...
GCP Data Engineer
Technical Skills</b></p> <p> 10+ years of experience with Data Warehouse / Data Platforms</p> <p> 5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data and SQL. Experience collaborating and...
Lead AWS Data Engineer
Data Engineer with AWS Glue Lambda SQLPython DovpsRedshift. Perform data cleansingdata validation etc. Must have working knowledge indesigning and implementing data pipelines on any of the cloudproviders (AWS is preferred). Must be able to work with largevolumes of data coming from various sources. ...
Data Engineer
As a Data Engineer, you will be responsible forbuilding and maintaining the systems and processes that allow for theextraction, transformation, and loading (ETL) of data from various sources intoour Azure-based data ecosystem. Azure DevOps Engineer with D365 and F&Oexperience. ...
Software Engineer, Data Transformation Movement
The Data Transformation and Movement team operates the critical infrastructure that powers near-realtime and batch data processing at Stripe. Data Transformation builds and operates the transformation abstractions and infrastructure that support frictionless data development across the board, sub-mi...