Senior Data Engineer (AWS/Python) to build lakehouse pipelines and revenue data products
S.i. Systems • Calgary
Il y a plus de 30 jours
Type de contrat
- Temps plein
Description de poste
Job Title: Senior Data Engineer (AWS/Python) to build lakehouse pipelines for airline customer and revenue data products
Overview: Our client seeks Senior Data Engineers to design and implement production-grade data pipelines in AWS lakehouse supporting Customer, Revenue, and Operations ° initiatives.
Responsibilities:
- Build scalable data ingestion pipelines from external sources
- Implement CDC patterns for operational system integration
- Develop PySpark transformations for Customer and Revenue products
- Optimize existing Redshift/Matillion workloads
- Create automated data quality monitoring and alerting
- Support Revenue production deployment
Must Haves:
- 7+ years building production data pipelines in cloud environments
- Expert-level Python and SQL for data transformation
- Hands-on experience with AWS Glue, EMR, Lambda, and Step Functions
- Strong background in Spark/PySpark for large-scale data processing
- Experience building CDC pipelines and real-time data ingestion
- Track record delivering data products from ingestion to consumption
Nice to Haves:
- Airline industry experience with reservation, revenue, or operational data
- Infrastructure as Code (Terraform/CloudFormation)
- Experience with streaming platforms (Kafka, Kinesis)