Data Engineer
1 month ago
We are a growing grocery app looking for an experienced Data Engineer to join our data team. As a key member of our team, you'll be responsible for building and maintaining our data infrastructure, optimizing our data processes, and ensuring data quality and accessibility across the organization.
Key Responsibilities:
1. Advanced Airflow Environment Development
- Build and maintain a robust Airflow environment to run scripts for creating datamarts and automating data logic
- Implement best practices including CI/CD, linter checking, secret key management, and Airflow variables
- Set up and manage Docker-based Airflow deployments for improved isolation and portability
- Configure and optimize Kubernetes executor for scalable and efficient task execution
- Implement advanced scheduling techniques, including dynamic task generation and cross-DAG dependencies
- Set up comprehensive monitoring and alerting for the Airflow environment
- Implement effective logging strategies for improved debuggability
- Ensure high availability and fault tolerance of the Airflow cluster
2. Data Pipeline Migration and Optimization
- Migrate existing data team scripts from the old Airflow environment to the new one
- Improve script quality and optimize performance during the migration process
- Implement data quality checks and SLAs for critical pipelines
3. Looker Governance and Optimization
- Manage Looker Enterprise implementation
- Develop and implement a strategy for tailored access to specific Looker explores for different teams
- Optimize Looker performance and ensure proper data governance
- Set up and maintain LookML CI/CD pipelines
4. Web Scraping Projects
- Design and implement various web scraping projects to collect relevant external data
- Ensure the quality and reliability of scraped data
- Implement robust error handling and retry mechanisms for web scraping pipelines
5. General Data Engineering Tasks
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements
- Design, build, and maintain scalable data pipelines and ETL processes
- Build and maintain cloud-based data warehouses (e.g., BigQuery), including schema design, optimization, and management
- Implement data modeling best practices for efficient querying and analysis
- Ensure data quality, reliability, and accessibility across the organization
- Optimize data warehouse performance and cost-efficiency
- Develop and maintain data documentation and metadata management systems
- Bachelor's degree in Computer Science, Data Science, or a related field
- 3+ years of experience in data engineering or a similar role
- Strong proficiency in Python and SQL
- Extensive experience with Apache Airflow, including advanced features and best practices
- Experience with Docker and container orchestration, preferably Kubernetes
- Proven experience in building and maintaining cloud data warehouses (especially BigQuery)
- Familiarity with Looker or similar BI tools
- Knowledge of web scraping techniques and tools
- Experience with cloud platforms (especially GCP)
- Strong problem-solving skills and attention to detail
- Excellent communication skills and ability to work in a team environment
-
IT Data Engineer
5 months ago
Jakarta, Indonesia PGI Data Full timeDevelop, construct, test and maintain cloud architectures to nurture data-focused products and internal tools Align said architecture with business requirements Acquire and integrate data from external sources Bachelor’s degree in any major (preferably in Informatics, Computer Science, Software Engineer) 2-4 years experience Good knowledge in Data...
-
Cloud Data Engineer
6 months ago
Jakarta, Indonesia IlmuOne Data Full timeWe are seeking a skilled Cloud Data Engineer to join our team. In this role, you will be responsible for designing, building, and maintaining cloud-based data infrastructure and services that enable our organization to process, store, and analyze large volumes of data. You will work closely with data scientists, data analysts, and other stakeholders to...
-
Cloud Data Engineer
6 months ago
Jakarta, Indonesia IlmuOne Data Full timeWe are seeking a skilled Cloud Data Engineer to join our team. In this role, you will be responsible for designing, building, and maintaining cloud-based data infrastructure and services that enable our organization to process, store, and analyze large volumes of data. You will work closely with data scientists, data analysts, and other stakeholders to...
-
Data Engineer
2 weeks ago
Jakarta, Indonesia IlmuOne Data Full time**Work Responsibilities**: - Create and maintain an optimal/stable data pipeline - Collect complex data for the functional and non-functional needs of the company & clients - Identify, design, and implement the development process which includes process automation, optimization of data provision, redesign of scalable infrastructure - Develop optimal...
-
Data Engineer
2 weeks ago
Jakarta, Indonesia IlmuOne Data Full time 8,000,000 - 17,000,000Work Responsibilities:Create and maintain an optimal/stable data pipelineCollect complex data for the functional and non-functional needs of the company & clientsIdentify, design, and implement the development process which includes process automation, optimization of data provision, redesign of scalable infrastructureDevelop optimal infrastructure for the...
-
Data Engineer
6 months ago
Jakarta, Indonesia IlmuOne Data Full timeResearch, build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using JavaScript or other languages - Contributes to team effort by accomplishing related results as needed. - Explore/learn new technologies that can complement or replace our current stack to improve it. -...
-
Data Engineer
6 months ago
Jakarta, Indonesia IlmuOne Data Full timeJob Description:Research, build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using JavaScript or other languagesContributes to team effort by accomplishing related results as needed.Explore/learn new technologies that can complement or replace our current stack to...
-
Data Engineer Expert
6 days ago
Jakarta, Jakarta, Indonesia IlmuOne Data Full timeAt IlmuOne Data, we are seeking a highly skilled Data Engineer Expert to join our team. This is an exceptional opportunity for a talented individual to take their career to the next level.About the RoleWe are looking for a motivated and detail-oriented Data Engineer who can design, build, and maintain scalable data infrastructure using JavaScript and other...
-
Senior Data Architect
2 weeks ago
Jakarta, Jakarta, Indonesia IlmuOne Data Full timeJob DescriptionWe are seeking a highly skilled Data Engineer to join our team at IlmuOne Data.About the RoleThis is an exciting opportunity to work with a talented team of professionals in a dynamic work environment. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines and architectures.Key...
-
Data Architect
3 weeks ago
Jakarta, Jakarta, Indonesia IlmuOne Data Full timeWe are seeking a Data Engineer to join our team at IlmuOne Data. In this role, you will be responsible for designing, building, and maintaining the data infrastructure that supports our business operations. The ideal candidate will have a strong understanding of cloud-based technologies, including GCP, AWS, and Azure, as well as experience with data...
-
IT Data Center Engineer, Aws Data Center Operation
5 months ago
Jakarta, Indonesia PT Amazon Data Srvcs Indonesia Full time2+ years of IT infrastructure operations experience. - Computer/Server Hardware Troubleshooting experience. - Basic understanding of Network operation / support experience - Written & verbal communication skills - This role involves covering 24x7 shift rotation Do you have a passion to install, deploy, service and maintain mission critical IT infrastructure...
-
Data Scientist
4 weeks ago
Jakarta, Indonesia IlmuOne Data Full time**Data Scientist Job Description**: **Work responsibilities**: - Assist in methodology research, data processing, and documentation to conduct independent self-guided exploration of research and frame open-ended industry questions, including, but not limited to: - Conducts statistical analyses to develop strategies. - Builds predictive models and...
-
Data Scientist
5 months ago
Jakarta, Indonesia IlmuOne Data Full time**Work responsibilities**: Assist in methodology research, data processing, and documentation to conduct independent self-guided exploration of research and frame open-ended industry questions, including, but not limited to: - Conducts statistical analyses to develop strategies, - Builds predictive models and machine-learning algorithms, - Identifies data...
-
Data Scientist
6 months ago
Jakarta, Indonesia IlmuOne Data Full time**Work responsibilities**: Assist in methodology research, data processing, and documentation to conduct independent self-guided exploration of research and frame open-ended industry questions, including, but not limited to: - Conducts statistical analyses to develop strategies, - Builds predictive models and machine-learning algorithms, - Identifies data...
-
Data Engineer
1 month ago
Jakarta, Indonesia PT. Kwadran Lima Mitra Kaya Solusi Full time**Role Description** The Data Engineer will be responsible for tasks such as data engineering, data modeling, ETL (Talend), data warehousing, and data analytics on a day-to-day basis. **Qualifications** - Education minimum Bachelor (S1) in Computer Science or Information Technology - Have at least **3 years** of experience as a **Data Engineer**,...
-
Data Systems Architect
5 days ago
Kota Administrasi Jakarta Pusat, Jakarta, Indonesia IlmuOne Data Full timeAre you a seasoned expert in building scalable data systems? We are seeking a highly skilled Data Systems Architect to join our team at IlmuOne Data.About the RoleIn this role, you will be responsible for designing and implementing robust data infrastructure to support our software applications. You will collaborate closely with cross-functional teams to...
-
Data Engineer
6 months ago
Jakarta, Indonesia PT ADI PERDANA NUSANTARA Full timePT ADI PERDANA NUSANTARA IS HIRING ! DATA ENGINEER - Bachelor's degree in Engineering (Computer/Telecommunication), Computer Science/Information Technology, Information System or equivalent - Have min. 2 years of experiece as Data Engineer - Translating data and business requirements into data engineering procesess that match business requirement - Provide...
-
Data Scientist
4 weeks ago
Jakarta, Indonesia IlmuOne Data Full time 7,000,000 - 12,000,000Data Scientist Job Description:Work responsibilities: Assist in methodology research, data processing, and documentation to conduct independent self-guided exploration of research and frame open-ended industry questions, including, but not limited to:Conducts statistical analyses to develop strategies.Builds predictive models and machine-learning...
-
Jakarta, Indonesia PT Amazon Data Services Indonesia Full time1. Demonstrate skills, competencies, and capabilities in leading and managing operation of critical facilities which require in-depth knowledge in electrical, mechanical, and control system 2. In depth knowledge of Data Center Facilities such as generators, chillers, cooling towers, air handling units, UPS and DRUPS, electrical sub distribution systems, fire...
-
Data Center Engineering Ops Engineer, Dceo, Aws Dceo
3 months ago
Jakarta, Indonesia PT Amazon Data Services Indonesia Full time3+ years of relevant work experience in maintaining electrical, mechanical, HVAC, fire systems experience in a data center or critical space facility - Ability to participate in a 24 x 7 rotating shift roster. - Understanding of the electrical and mechanical systems used in a data center environment, including but not limited to DRUPS, Transformers,...