Int Data Engineer to design, build and maintain data infrastructure
Our valued client is looking for an Int Data Engineer to design, build and maintain data infrastructure.
initial 3-6 month contract hybrid in Ottawa with possibility for conversion to full time permanent. (min 2 days a week onsite)
Responsibilities :
- Designing, building, and maintaining data infrastructure that supports the efficient extraction, transformation, and loading (ETL) of data from various sources.
- Develops ways to increase the usability of data through changes in business process and / or data cleansing techniques
- Design, build, and maintain data pipelines and ETL processes using tools such as Streamsets, Apache Spark, Hadoop, and other big data technologies.
- Develop and maintain data infrastructure including data warehouses and other data storage solutions to support business intelligence and reporting needs
- Design and implement data security measures to protect sensitive data.
- Develop and maintain data quality control processes to ensure the accuracy, reliability, and accessibility of data to all stakeholders.
- Monitors system performance of scheduled ETL batches, and streaming data process and ensures all systems are working at an acceptable standard and that data pipelines are scalable, repeatable, and secure.
- Performs data migration between development, UAT, and production systems, and plans and coordinates these data migrations.
- Analyzes and troubleshoots technical issues quickly to resolution, working with internal ITS sections and software vendors, when required.
Must Have Skills :
- 5+ years experience in a Data Analytics environment with progressively more technical responsibilities in an engineering role
- Designing and creating :
- ETL processes, data pipelines and workflows.
- Logical and physical data models using data modelling best practices.
- Develop scripts, applications and APISs to automate data processing tasks using programming languages such as SQL, Python, Java, Scala, shell scripting, JavaScript
- Designing, building, and supporting data warehouses, data hubs and other data storage and processing systems.
Nice to Have Skills :
- Experience with cloud computing platforms such as Azure and being familiar with setting up and managing cloud-based data storage and computing environments.
- Working with stream processing frameworks such as Apache Kafka or Streamsets
- Designing and implementing :
- Real-time data processing pipelines using these tools.
- Database solutions using technologies such as MySQL, PostgreSQL or SQL Server.
- Project implementation analysis and support in data management systems, data integrity and security as it relates to environmental business systems.
- Machine Learning concepts and tools (R, Python, Jupyter Notebook).
- Utilization of tools such as Apache Spark, Hadoop, and other big data technologies.
Forcepoint DLP (Data Loss Prevention) Engineer
As the world's leading mission capability integrator and transformative enterprise IT provider, we deliver trusted, highly differentiated solutions and technologies to protect our nation and allies. Provide troubleshooting and support of a complete DLP tool suite (Forcepoint) environment including c...
Senior Security Engineer specializing in encryption and cryptography to build automation within HSM, Certificate and Key Management
Senior Security Engineer specializing in encryption and cryptography to build automation within HSM, Certificate and Key Management. Experience building and operating. Experience developing policies, procedures related to cryptography,. ...
Bilingual Senior Specialist, Data Governance
Support the development of directives, standards and guidelines, define the roles and responsibilities of stakeholders and ensuring successful implementation and maintenance of an information governance framework according to the data lifecycle management framework. Provide functional/technical guid...
Records & Data Entry Assistant
The Records & Data Entry Assistant is responsible for data entry, maintaining the accuracy and integrity of computerized records systems and responding to requests and enquires. Ability to demonstrate attention to detail and accuracy. Ability to communicate with others to explain and share informati...
Data Engineer
Experience designing, architecting, and maintaining a data warehouse that seamlessly stitches together data from production databases, clickstream event data, and external APIs to serve teams of analysts. You’ll work to understand requirements, then design, deploy, test, and deploy data pipelines, a...
Senior SAP Quality Assurance Manager to build out a new QA practice within a SAP Center of Excellence bringing 12+ years progressive QA experience in manual and automated test environments.
Senior SAP Quality Assurance Manager to build out a new QA practice within a SAP Center of Excellence bringing 12+ years progressive QA experience in manual and automated test environments. Developing QA Strategies: You understand and have experience bringing SAP Quality Assurance Best Practices to ...
Senior Data Engineer
Engineer and enhance data pipelines, ETL processes, and data cleaning mechanisms to support a comprehensive cybersecurity data analysis system. We are on the lookout for a talented Senior Data Engineer to contribute significantly to our advanced data collection, processing, and analysis system which...
Data Engineer (Analyst)
A successful candidate will have Cloud experience and strong python and SQL experience. The Select Group is seeking a Data Analyst for their top telecommunication client. Remote candidates are welcomed to apply. For this role, the Analyst will join a team that manages the data lakes within the Googl...
Data Engineering Manager - Data Platform & Services
Provide leadership and guidance to technical leads, software engineers and data scientist, ensuring alignment with company goals and maintaining platform performance, reliability, and security and cost efficiency. We’re making major investments to enhance our customer experience by making these serv...
Senior SAP Enterprise Threat Detection (ETD) Consultant to design effective security strategies and to optimize the solutions monitoring capabilities for a public sector client.
Senior SAP Enterprise Threat Detection (ETD) Consultant to design effective security strategies and to optimize the solutions monitoring capabilities. The client recently implemented and deployed SAP ETD into the their large-scale, complex, environment but have identified a gap with the monitoring c...