Big Data QE Engineer

Job Description

Big Data QE Engineer

Kitchener, ON

Work Model: Hybrid (2 days in office)

Experience Required: 6–8 Years

Role Description

We are seeking a motivated Big Data QA Engineer / Data Tester with experience in data testing and a passion for the Property & Casualty (P&C) Insurance domain. The ideal candidate will be responsible for supporting data testing initiatives, validating data quality, and ensuring solutions meet design specifications, standards, and business requirements within an Agile environment.

Key Responsibilities

• Develop and execute test strategies aligned with design specifications, standards, and user requirements.

• Perform ETL and database testing, data migration testing, and report validation.

• Write and execute complex SQL queries for data analysis and validation.

• Execute and validate queries in GCP BigQuery.

• Design and execute test cases in a Big Data environment.

• Analyze data and validate data quality across various data sources.

• Collaborate with Agile teams to test components iteratively throughout the development lifecycle.

Required Skills

• 6–8 years of experience in ETL Testing, SQL, Big Data, and Python.

• Hands-on experience with:

• ETL and Database Testing

• Data Migration Testing

• Report Validation

• Strong knowledge of GCP BigQuery.

• Experience with PySpark and/or Python.

• Expert-level SQL skills, including writing and understanding complex queries.

• Advanced knowledge of Big Data technologies, including:

• Hadoop

• Hive

• Impala

• HBase

• Pentaho

• Experience working in Agile project environments.

Preferred Skills

• Exposure to the Property & Casualty (P&C) Insurance domain, specifically Property and Auto Insurance.

• Knowledge of Guidewire.

Requirements
Top 3 Required Skills: 1. IBM Financial transaction 2. Payment flow 3. Support Modernization Detailed Job Description: • Design, develop, and maintain applications built on IBM Financial Transaction Manager (FTM) to support core payments processing. • Contribute to the development of payment flows supporting transaction processing. • Build and support integrations between FTM and upstream/downstream systems using enterprise integration patterns. • Participate in the design, development, testing, deployment, and production support. • Troubleshoot and resolve application and integration issues in a complex, regulated environment. • Collaborate with architecture, QA, and operations teams to ensure platform stability, scalability, and performance. • Support modernization initiatives and enhancements to existing payment hub capabilities. • Produce clear technical documentation and participate in code reviews and knowledge sharing.