Senior AWS Data Engineer/Architect

Our client is looking for a Senior Data Engineer/Architect to provide technical leadership and guidance migrating an on-prem data warehouse to a data like architecture on AWS.

Fully remote in Canada, 17-35 hours/week, contract length: 4-5 months.



Background and Project Description:

• This project involves migrating on-premises SQL Server/SSIS based data warehouse to a modern AWS-based data Lakehouse architecture. The team has done exploratory and pilot work to establish a target architecture utilizing AWS technologies including DMS, S3, AWS Glue, dbt, Apache Airflow, Redshift, CloudFormation, and Cloudwatch.
• Data sources to be captured and ingested include Oracle, SQL Server databases, and a variety of 3rd party APIs.
• Data consumers of the data lake will include data science and business intelligence tools and processes from on-prem and cloud infrastructure.

General Roles and Responsibilities and Deliverables:

• Developing data pipeline and orchestration design patterns using AWS tools including DMS, S3, AWS Glue, Managed Workflows for Apache Airflow and dbt
• Develop optimal patterns for data pipeline and data quality monitoring and alerting
• Conversion of SQL Server and SSIS data warehouse data models and code to dbt, Airflow and Redshift data models and code
• Ensure security and data access best practices (IAM roles, encryption, credential management) and adherence to TSBC IT security process and TSBC data governance policy
• Collaborate with IT and Data Management team to design a CI/CD deployment system for managing development and production AWS data lake accounts

Education and Experience

• Degree in computer science, computer engineering, system engineering, big data and statistics or equivalent work experience
• 6+ years data engineering, applied data science, or software engineering experience
• Experience with large-scale data migrations or cloud modernization projects
• Capability to architect highly scalable distributed systems, using AWS tools.
• Familiarity with DevOps best practices and AI code assistance tools
• Understands data pipeline technologies and tools and have experience building high-performance data systems.
• Extensive knowledge in SQL, python and dbt
• Expert knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases.
• Experience using AWS technologies including S3, Redshift, AWS Glue, CloudFormation, CloudWatch
• Experience with integration of data from multiple data sources and multiple styles of APIs
• Experience working with SQL databases such as Oracle, SQL Server, MySQL, Redshift
• Knowledge of various ETL techniques and frameworks

For more current job opportunities, Follow MatchBox at linkedin.com/company/matchbox-recruitment/

_________________________

About MatchBox

Headquartered in downtown Vancouver and serving clients nationally, MatchBox is one of the leading recruitment and solutions firms that specializes in the fields of IT & Technology, Engineering & Technical, Real Estate & Construction, and Accounting & Finance. We offer unrivaled expertise with our team's extensive experience in the recruitment and professional services industry. We are dedicated to building great connections and creating strong opportunities within the workforce and the labor market.

Equal Opportunities

MatchBox is committed to providing equal opportunities for all applicants. We welcome and encourage applications from people of all backgrounds, including members of minority groups such as racialized individuals, people with disabilities, LGBTQ+ individuals, and Indigenous peoples. It is a priority for us that all candidates are treated fairly and without discrimination. Our recruitment teams receive ongoing training on using objective criteria for evaluating candidates and other related topics to create an inclusive and welcoming environment for all.

Work Permits

Please note that we require all interested candidates to obtain the necessary work permits from the Government of Canada prior to submitting an application. This position is open exclusively to individuals residing in Canada and legally authorized to work in Canada. Applications not meeting these criteria will not be considered.

Further Consideration

We receive a high volume of applications and are only able to contact candidates who are selected for further consideration.

Find out more at www.matchboxhr.com