Software Engineer III - Metadata Management & Data Discovery

Description

:

Role summary

We are seeking a highly technical Software Engineer III – Metadata Management & Data Discovery to help lead the design, engineering, and delivery of enterprise metadata harvesting capabilities across cloud, multi-cloud, and legacy on-premises data platforms.

This role will be responsible for ideating, designing, building, and operationalizing scalable metadata harvesting solutions that enable automated discovery, cataloging, lineage, classification, and governance of enterprise data assets. The successful candidate will support the Data Discovery and Metadata Harvesting function by building reusable patterns and integrations that harvest structured and semi-structured metadata from platforms such as Databricks Unity Catalog, Azure ADLS, Azure Synapse, Snowflake, AWS, Google Cloud, Oracle, SQL Server, DB2, Teradata, mainframe files, SFTP/network shares, and other enterprise data stores.

The role will be central to advancing Data Compass / Collibra as the enterprise metadata system of record and will partner closely with engineering, cloud infrastructure, architecture, security, governance, and data platform teams to ensure metadata is harvested, normalized, enriched, reconciled, and maintained in a trusted and audit-ready manner.

Key Responsibilities

• Lead the technical design and implementation of enterprise metadata harvesting solutions across cloud, multi-cloud, and on-premises platforms.
• Build automated harvesting pipelines and connectors to capture technical, operational, business, lineage, classification, ownership, and stewardship metadata.
• Design reusable ingestion patterns using Collibra EDGE connectors, REST APIs, JDBC/ODBC, event-driven approaches, file-based ingestion, and custom adapters where needed.
• Engineer metadata harvesting solutions for Databricks Unity Catalog, including catalog, schema, table, view, column, permission, tag, lineage, and operational metadata extraction.
• Develop scalable harvesting patterns for Azure data platforms, including ADLS Gen2, Azure SQL, Synapse, Azure Data Factory, Key Vault, private endpoints, service principals, and secure connectivity.
• Extend metadata harvesting capabilities across Snowflake, AWS, Google Cloud, and major on-premises data stores.
• Design and implement reconciliation controls to compare source platform inventories against harvested assets in Data Compass / Collibra.
• Define and maintain asset mapping, attribute mapping, classification rules, lineage stitching logic, enrichment patterns, and metadata quality checks.
• Partner with cloud infrastructure, IAM, security, and networking teams to establish secure access patterns for harvesting across environments.
• Support environment promotion from DEV/SIT/PAT/PROD with proper controls, monitoring, alerting, logging, and operational readiness.
• Build monitoring and support capabilities for harvesting jobs, including failure alerts, retry logic, health checks, lineage validation, and incremental harvesting.
• Collaborate with governance and business teams to ensure metadata is complete, trusted, actionable, and aligned to enterprise data standards.
• Create technical documentation, architecture diagrams, implementation guides, runbooks, and reusable patterns for onboarding new sources.
• Mentor engineers and analysts on metadata harvesting engineering practices, source onboarding, validation, and automation.
• Help advance the broader Data Discovery roadmap, including Unity Catalog harvesting, Data Compass integration, lineage expansion, AI/ML metadata harvesting, and critical system metadata coverage.

Required Experience/Education

• Undergraduate degree, Post Graduate degree or Technical Certificate
• Strong academic background (e.g., computer science, engineering). Graduate degree nice to have
• 5-7 years relevant experience
• Strong hands-on experience designing and building data engineering, metadata management, data catalog, or data governance technology solutions.
• Strong programming experience with Python, SQL, and API-based integration patterns.
• Deep experience with Databricks, Unity Catalog, Delta Lake, Spark/PySpark, and cloud data platform engineering.
• Strong understanding of Azure infrastructure, including ADLS Gen2, Synapse, Azure SQL, Azure Data Factory, Entra ID, service principals, managed identities, Key Vault, private endpoints, VNets, NSGs, and RBAC.
• Experience integrating with or engineering around Collibra, Data Compass, Collibra EDGE, Purview, Alation, Informatica, or similar catalog/governance platforms.
• Experience with metadata extraction from structured and semi-structured sources, including databases, data lakes, files, schemas, views, tables, columns, tags, and operational metadata.
• Experience with cloud data platforms such as Snowflake, AWS, and/or Google Cloud Platform.
• Working knowledge of legacy data platforms such as Oracle, SQL Server, DB2, Teradata, mainframe files, flat files, and SFTP/network-share based data stores.
• Strong understanding of data governance concepts, including lineage, classification, ownership, stewardship, business glossary, data quality, and metadata lifecycle management.
• Experience designing secure, scalable, supportable engineering solutions with CI/CD, Git, logging, monitoring, automated testing, and production support practices.
• Ability to work across technical and non-technical stakeholders and translate complex metadata/platform topics into clear execution plans.

Who We Are:

TD is one of the world's leading global financial institutions and is the fifth largest bank in North America by branches/stores. Every day, we strive to make every interaction, product, and experience remarkably human and refreshingly simple for over 27 million households and businesses in Canada, the United States and around the world. More than 95,000 TD colleagues bring their skills, talent, and creativity to foster deeper relationships, ensure disciplined execution, and build a simpler, faster banking experience. TD is deeply committed to being a leader in client experience, that is why we believe that all colleagues, no matter where they work, are client facing. Together, we are reimagining what banking can be for our clients, colleagues and communities.

Our Total Rewards Package
Our Total Rewards package reflects the investments we make in our colleagues to help them and their families achieve their financial, physical, and mental well-being goals. Total Rewards at TD includes a base salary, variable compensation, and several other key plans such as health and well-being benefits, savings and retirement programs, paid time off, banking benefits and discounts, career development, and reward and recognition programs.

Additional Information:
We’re delighted that you’re considering building a career with TD. Through regular development conversations, training programs, and a competitive benefits plan, we’re committed to providing the support our colleagues need to thrive both at work and at home.

Please be advised that this job opportunity is subject to provincial regulation for employment purposes. It is imperative to acknowledge that each province or territory within the jurisdiction of Canada may have its own set of regulations, requirements.

Colleague Development

If you’re interested in a specific career path or are looking to build certain skills, we want to help you succeed. You’ll have regular career, development, and performance conversations with your manager, as well as access to an online learning platform and a variety of mentoring programs to help you unlock future opportunities.

If you’re passionate about helping clients and building deep, lasting relationships, TD offers diverse career paths where you can grow your expertise and make a meaningful impact.

We're committed to your success and foster a respectful workplace where diverse perspectives are valued, everyone has fair opportunities to grow, and you can unlock your full potential to achieve your career goals. Here at TD, we hire and develop the best.

Training & Onboarding
We will provide training and onboarding sessions to ensure that you’ve got everything you need to succeed in your new role.

Interview Process
We’ll reach out to candidates of interest to schedule an interview. We do our best to communicate outcomes to all applicants by email or phone call.

Accommodation
Your accessibility is important to us. Please let us know if you’d like accommodations (including accessible meeting rooms, captioning for virtual interviews, etc.) to help us remove barriers so that you can participate throughout the interview process.

We look forward to hearing from you!

Language Requirement (Quebec only):

Sans Objet