DATA ENGINEER

Job Description

• Build and maintain data pipelines from various data sources.
• Develop and maintain web crawling/scraping processes to collect data.
• Perform ETL (Extract, Transform, Load) processes to prepare data for further use.
• Clean, transform, and validate data to ensure good data quality.
• Develop scripts to process documents and generate metadata from PDF files.
• Monitor and troubleshoot data crawling and processing processes.
• Automate data processing workflows using tools such as Airflow, n8n, or similar tools.
• Collaborate with the team to support data needs for applications, analytics, and AI-related projects.

Kualifikasi

• Bachelor’s degree in computer science, mathematics, statistics, or related fields.
• Minimum 1 Years experience as Data engineer, Data Processing, Web Scarping, or a related field
• Proficient in SQL and at least one programming language, preferably Python.
• Familiar with web crawling/scraping and data cleaning.
• Familiar with ETL/data pipeline processes.
• Familiarity with Airflow, n8n, or similar tools.
• Familiar with relational databases such as PostgreSQL or MySQL.
• Experience with PDF/document processing is a plus.
• Strong analytical and problem-solving skills
• Able to work independently and within a team
• Willing to continuously learn and grow

Skills: Database Systems, Data Integration, Microsoft SQL Server, Python, ETL, Redis, Hadoop, MongoDB, Data Processing, Data Engineering