Lead Data Engineering Analyst

Design and develop state-of-the-art data analytics solutions for business problems

• Architect Big Data analytics frameworks and solutions to solve data driven business problems. Evaluate Tiger's existing data and analytics platforms and develop core product strategy by customizing and standardizing tools and components.
• Study the existing data architecture for client’s big data platforms to see if the data retrieval latency and velocity is acceptable.

Monitor, collect, and review business requirements, and related technology solutions. Use this information to suggest design of new big data frameworks and Data Lake solutions.

Establish platform governance via data-flow-diagrams, and system reviews for all new initiatives and existing programs.

Solve problems of large dimensionality (terabytes of data storage) in a computationally efficient and statistically effective manner use codes and modules that reduce the time to read the data and also allow machine learning algorithm to run in real time.

Collect the requirement on the acceptable latency and test against that.

• Work on advancing emerging platform features for artificial intelligence, machine learning, advance analytics, cloud data warehousing, and metadata management by working closely with business, technical and data science teams to gather, analyze and understand business requirements, translate activities and objectives into analytical models and algorithms for application in production scale environment.
• Design the data pipelines so data can be used for Data Science, Machine Learning, Artificial Intelligence, Operational Research techniques.

Drive personalization, real-time decision-making, causal inference and predictive analytics capabilities through the application of Machine Learning, Deep Learning, simulation and AI processes in an agile development framework.

Research the academic and industrial best practices in the Big Data Analytics field to understand the trends in increasing data volume (size of data), data velocity (speed of data access), data variety (types of data available to businesses).

Advise clients on the latest tools and technologies in this space

Build quantitative programs using latest analytics tools

Analyze sample data sets given by the client to understand the size and type of data (structured or unstructured, text, image, video).

Then provide options on the suitable big data frameworks and databases to use.

Survey business users and IT teams to identify, analyze, collect, transform, and document data sets that can be used to drive business insights.

Extract, transform and organize the data into datasets using tools like SQL

• Test solution algorithms on real-time data processing systems and benchmark performance using R, Python, etc.
• Validate the results of the model by developing testing framework using Hadoop, Python, etc.
• Build large scale fault tolerant enterprise applications using Hadoop and Big Data open source solutions such as : MR, Hive, Pig, HBase, Spark
• Define the data architecture and work on end to end holistic data solutions that include OLTP data stores (SQL data stores and others) , NoSQL data stores (Cassandra and others ), messaging and data movement products (such as Kafka, ActiveMQ, and others), search engine products (such as Elastic Search, and others), as well as OLAP data stores (such as Hadoop, HBase, Teradata, and others).
• Develop new data sources by analyzing similar use cases from the Tiger’s case studies and accelerator poos. Simulate market scenarios using Python or other programming tools to create and evaluate new additions to enhance data assets.
• Work with multi-functional teams to access data elements, understand the data being analyzed, and identify improvement opportunities for data ingestion process.
• Develop repeatable testing strategies for measuring results produced by analytics models.
• Operationalizing an enterprise data governance strategy. Aligning policies and processes to support data strategy, data quality, regulations, and latency.

Supervise teams of Data Analysts and Data Engineers

• Understand the business context and suggest analytical solutions using Big data architectures based on industry knowledge
• Identify the parts of a data driven business problem that require Big Data solutions. Develop flow charts and solution diagrams proposal.
• Then discuss the proposal with client’s cross-functional teams to divide project goals into specific tasks involving business analysis, data analysis and data engineering;
• Work with client’s IT stakeholders to deploy the developed solution and make sure it can be implemented in production by the client engineering team
• Lead all data collection, validation, and analysis activities including risk, spend, commodity and supplier information.
• Deliver datasets with the appropriate characteristics for the desired usage patterns and use cases.
• Benchmark the developed solution for data consistency, data latency, data quality and deploy them into production.

Job Requirements :

Education Qualification : Bachelor's Degree

Salaries may increase annually based on performance. Additional bonuses may be issued at the company's discretion and comparative market trends.

If issued, the bonus may be in the form of Cash or ESOPS. COLA may be added as well at the company's discretion.