Data Platform Developer

Overview

We're seeking a talented Data Platform Developer to join our Platform Engineering team and architect the data foundation that will power Newforma's next generation of AI-driven capabilities and analytics. You'll design and implement modern data architectures including medallion / lakehouse patterns, build event-driven data pipelines that process billions of project documents and communications in real-time, and create the analytics infrastructure that enables both business intelligence and AI / ML initiatives. This is a foundational role at an exciting time—as we migrate to AWS and invest heavily in AI, you'll establish the data practices and infrastructure that will serve the company for years to come.

Newforma manages billions of emails, documents, RFIs, submittals, drawings, and project files for thousands of construction projects worldwide. This rich dataset represents an incredible opportunity for AI-powered insights, intelligent automation, and advanced analytics. You'll build the data infrastructure to unlock this potential, creating pipelines that transform raw project data into clean, structured, and AI-ready datasets while also enabling real-time analytics and business intelligence. Working closely with our Director of AI Engineering and Platform Engineering team, you'll establish data architecture patterns that support everything from semantic search and RAG systems to executive dashboards and predictive analytics.

In this role, your responsibilities will include :

Data Architecture & Strategy

• Design and implement medallion architecture (bronze, silver, gold layers) or lakehouse patterns on AWS to organize and transform data at scale
• Establish data modeling standards, governance practices, and quality frameworks across the organization
• Define data retention, archival, and lifecycle management policies for massive volumes of project data
• Create reference architectures and best practices for data engineering across teams
• Partner with the Director of AI Engineering to design data pipelines optimized for AI / ML workloads including vector embeddings and model training
• Work with the Lead Software Architect to ensure data architecture aligns with overall platform strategy
• Design data schemas and structures that support both analytical queries and AI applications
• Design and implement event-driven data architectures using AWS EventBridge, Kinesis, MSK (Kafka), SNS, and SQS
• Build real-time data streaming pipelines that capture, process, and route project events across the platform
• Architect event schemas and patterns for domain events (document uploads, email filing, RFI submissions, etc.)
• Implement change data capture (CDC) patterns to stream database changes to data lakes and analytics systems
• Design event-driven workflows that trigger AI processing, notifications, and downstream system updates
• Establish event governance including versioning, documentation, and monitoring
• Optimize event processing for low latency and high throughput at scale
• Build robust, scalable ETL / ELT pipelines using AWS Glue, Step Functions, Lambda, and EMR
• Develop data transformation jobs that cleanse, enrich, and structure unstructured project data
• Implement data quality checks, validation rules, and monitoring throughout pipelines
• Create reusable pipeline components and frameworks that teams can leverage
• Optimize pipeline performance and cost efficiency for processing billions of documents
• Handle diverse data formats including emails, PDFs, CAD drawings, images, and structured databases
• Implement data lineage tracking and metadata management
• Design and build data warehouses and data marts using Amazon Redshift, Athena, or similar technologies
• Create dimensional models and star schemas optimized for analytical queries
• Build datasets and aggregations that power executive dashboards and operational reports
• Implement BI solutions using tools like QuickSight, Tableau, PowerBI, or similar platforms
• Partner with product and business teams to understand analytics requirements and deliver insights
• Create self-service analytics capabilities that empower teams to explore data independently
• Establish KPIs, metrics, and reporting frameworks for product and business analytics

AI / ML Data Infrastructure

Data Operations & Monitoring

Requirements

Nice to have qualifications

#J-18808-Ljbffr