Data Engineer

Join to apply for the Data Engineer role at Integrity Asia .

Get AI-powered advice on this job and more exclusive features.

Are you a passionate data engineer ready to build the infrastructure that powers our insights and decisions? As a Data Engineer , you will support the Data Scientist team by designing and building robust data pipelines, ensuring clean, structured, and accessible data for analytics, modeling, and decision-making processes.

What You’ll Do :

• Build and Maintain Data Pipelines
• Develop, test, and maintain scalable ETL / ELT pipelines that collect and process raw data from various sources.
• Automate data workflows to ensure timely and accurate data availability.
• Integrate various data formats and APIs into centralized systems.
• Develop and Manage Data Crawlers
• Build and maintain scalable web crawlers and data scraping engines to collect information from public or internal websites and APIs.
• Ensure compliance with ethical and legal web scraping practices (robots.txt, rate limits, etc.).
• Monitor crawler performance and implement logging, error handling, and retry mechanisms.
• Data Preparation and Transformation
• Clean and preprocess raw data to ensure consistency and usability.
• Create structured datasets from semi-structured and unstructured data.
• Support feature engineering and data modeling needs of Data Scientists.
• Data Infrastructure and Tools Management
• Manage and maintain databases, data lakes, or data warehouses (e.g., SQL, NoSQL, cloud platforms).
• Set up monitoring and logging for data systems to ensure performance and reliability.
• Ensure systems scale with data volume and processing demand.
• Ensure metadata management and proper data cataloging.
• Collaboration and Support
• Work closely with Data Scientists to understand project data needs and deliver appropriate data sets.
• Partner with software engineers, developers, and business stakeholders for end-to-end data solutions.
• Provide documentation, version control, and data lineage for transparency and reuse.

Who We Need :

Technical Skills Needed :

If you meet these requirements and are looking for an exciting opportunity within a successful, growing company, then this role is for you!

Integrity Indonesia is an equal opportunity employer.

Integrity Indonesia is a foreign investment company operating since 2001 in Indonesia, with affiliates in Malaysia and Thailand. Our business involves services related to the Prevention, Detection, and Investigation of risk.

As a rapidly expanding company with a robust business plan, Integrity Indonesia seeks self-motivated, results-oriented individuals with solid qualifications who enjoy challenges and growth opportunities.

Seniority level

Employment type

Job function

Industries

#J-18808-Ljbffr