Data Engineer
On behalf of our client, Procom Consultants Group is currently looking for an Intermediate Data Engineer.
Requested resources to provide developer expertise on coding of data transformation pipelines and systems integration tasks relating to enterprise data management applications that span across different business areas and platforms.
Data Engineer Job Details
- Creating mapping documentation of data elements from source to target and liaising with other Data Engineers, Data Architects and SMEs.
- Developing & testing Data Transformation pipelines by leveraging the latest Big Data tools and technologies
- Collaborate with upstream data teams, develop data validation and test plans, performing user acceptance testing, and provide feedback to development and sustainment teams
- Conducting analysis for moderate to complex data requests, defining data fields and determining data availability, developing information layout, format and interactivity.
Presenting findings and providing clarification.
- Act as a lead developer to implement code changes in various environments and platforms leveraging our DevOps & DataOps
- Work closely with the testing team to assist with issues raised in user acceptance testing (UAT) and Business Value Testing (BVT)
Deliverables :
Fully implement the new data format across all IM supported platforms including
Big Data (Hadoop), Enterprise Data Warehouse (EDW) and Operational Data Store
ODS)
Data Engineer Mandatory Skills
Communication & Problem Solving Skills :
- Communicating & presenting solutions to internal developers
- Ability to apply critical thinking skills to develop efficient code, data transformation logic and solution design
- Experience and / or understanding of Agile Methodologies
Technical Skills :
- Advanced Database Skills & SQL
- Experience with or exposure to Big Data platforms, ideally exposure to Hadoop ecosystem (HDFS, Apache Hive, Apache Spark, SparkSQL)
- Extensive programming experience in either Java, C++ or preferably Scala, IntelliJ, Jenkins
- Working experience with Apache Spark Data Processing Framework, Parquet data format
- Good understanding of Relational (Oracle / DB2) and Hadoop Big Data platforms
- Experience working with Jira
Data Engineer Start Date
Oct 24, 2022
Data Engineer Assignment Duration
March 31 2023
Data Engineer Working Location
Fully remote but PST hours
Data Engineer
We are actively seeking a Data Engineer to join our team! As a key member of the Data Systems team, you will design, deploy, and maintain robust data management systems and software applications that enable rapid access, interpretation, and action on information. Create reliable and secure connectio...
Work from Home Big Data Engineer for AI Training
Are you an experienced software engineer who would like to lend your coding expertise to train AI models?. PLEASE NOTE: We collect, retain and use personal data for our professional business purposes, including notifying you of opportunities that may be of interest and sharing with our affiliates. W...
data engineer
Operate database management systems to analyze data. Develop and implement data administration policy, standards and models. ...
Staff Software Engineer with Java (Springboot) to build out features on a Master Data Management product from start to finish.
Staff Software Engineer with Java (Springboot) to build out features on a Master Data Management product from start to finish. ...
Expert Software Engineer (Privacy Data)
We are looking for a data engineer to become a cornerstone of our data management strategy. At this scale, the challenges we face empower our engineers to flex their creative engineering muscles. We work with our customers to ship data forward applications, ensure compliance of data produced and ove...
Software Engineer Intern (Data-TnS-Eng-Core Safety) - 2025 Start (BS/MS)
The Trust and Safety(TnS) engineering team is responsible for protecting our users from harmful content and abusive behaviors. With the continuous efforts of our trust and safety engineering team, TikTok can provide the best user experience and bring joy to everyone in the world. Responsibilities- C...
Big Data Engineer for AI Training
Are you an experienced software engineer who would like to lend your coding expertise to train AI models?. PLEASE NOTE: We collect, retain and use personal data for our professional business purposes, including notifying you of opportunities that may be of interest and sharing with our affiliates. W...
Senior Consultant/ Manager - Microsoft Azure & Microsoft Fabric Data Engineer
Hands on experience with data processing and analytics using Azure Data Factory, Azure Synapse, Azure Data Lake or Azure Databricks is a plus. Contribute to the design and development of enterprise-level data solutions, including data warehouses, data lakes, and real-time analytics platforms, using ...
Data Engineer
As a Data Engineer, you will be instrumental in moulding EAX's data solutions, including structure, validation, documentation, storage, and retrieval. Partnering with product owners, designers, DDs, QV, and other engineers, you will design unified data models that support the current needs of our ga...
2 Senior Data Engineers to build data pipelines, integrate data sources, and clean and transform data for a data ecosystem modernization
Our public sector client is seeking 2 Data Engineers to to build data pipelines, integrate data sources, and clean and transform data for a data ecosystem modernization project. Azure Data Lake, Azure Data Factory, and Databricks (DBT is a plus). Experience with big data technologies such as. ...