Data Engineer with Python

4 months ago


Jakarta, Indonesia Unison Consulting Pte Ltd Full time

Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform.
- Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale.
- Optimize and troubleshoot AWS Glue jobs for performance and reliability.
- Utilize Python and PySpark to efficiently handle large volumes of data during the ingestion process.
- Design and implement scalable data processing solutions using PySpark to transform raw data into a structured and usable format.
- Create and maintain ETL processes using Python and PySpark to move and transform data between different systems.
- Optimize ETL workflows for performance and efficiency.
- Collaborate with data architects to design and implement data models that support business requirements.
- Ensure data structures are optimized for analytics and reporting.
- Work with distributed computing frameworks, such as Apache Spark, to process and analyze large-scale datasets.
- Manage and optimize databases, both SQL and NoSQL, to support data storage and retrieval needs.
- Implement indexing, partitioning, and other database optimization techniques.
- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver effective solutions.
- Implement monitoring solutions to track data pipeline performance and proactively identify and address issues
- Ensure compliance with data privacy regulations and company policies.
- Stay abreast of industry trends and advancements in data engineering, Python, and PySpark.

**Requirements**:

- Proficiency in Python and PySpark.
- Strong knowledge of data engineering concepts and best practices.
- Hands-on experience with AWS Glue and other AWS services.
- Experience with big data technologies and distributed computing.
- Familiarity with database management systems (SQL and NoSQL).
- Understanding of ETL processes and data modeling.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Bachelor's degree in Computer Science, Information Technology, or a related field.


  • Big Data Engineer

    3 months ago


    Jakarta, Indonesia PGI Data Full time

    Analyze Business requirement Develop ETL Script Develop Report Develop API & Microservices Bug Fix existing Script/report Monitor Job before handover to Ops team. Bachelor’s degree in any major (preferably in Informatics, Computer Science, Software Engineer) Having mínimal 2 year(s) experience as a Data Engineer or similar software engineering...

  • Data Engineer

    3 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    Research, build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using JavaScript or other languages - Contributes to team effort by accomplishing related results as needed. - Explore/learn new technologies that can complement or replace our current stack to improve it. -...

  • Data Engineer

    3 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    Job Description:Research, build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using JavaScript or other languagesContributes to team effort by accomplishing related results as needed.Explore/learn new technologies that can complement or replace our current stack to...

  • Python Data Scientist

    3 months ago


    Jakarta, Indonesia Nucleus Health Full time

    **Responsibilities**: - Write effective, high-quality Python code to solve problems - Clearly explain the reasoning behind the code in Jupyter notebooks - Debug code and produce thorough documentation - Using free datasets (such as Kaggle, the UN, the US government, etc.), use your data analytic skills to formulate and respond to important business issues -...

  • IT Data Engineer

    3 months ago


    Jakarta, Indonesia PGI Data Full time

    Develop, construct, test and maintain cloud architectures to nurture data-focused products and internal tools Align said architecture with business requirements Acquire and integrate data from external sources Bachelor’s degree in any major (preferably in Informatics, Computer Science, Software Engineer) 2-4 years experience Good knowledge in Data...

  • Data Engineer

    3 months ago


    Jakarta, Indonesia Group Hijra (Hijra Bank, ALAMI P2PL) Full time

    Hijra (previously known as the ALAMI Group) is a financial technology company that follows sharia principles. It was founded in 2018 by Dima Djani, Harza Sandityo, and Bembi Juniar. The company offers a range of services, including a peer-to-peer lending platform, mobile banking app, and mortgage all of which are based on sharia principles. Hijra has a...

  • Data Engineer

    2 months ago


    Jakarta, Indonesia LippoInsurance Full time

    **Responsibilities**: **Requirements**: - Bachelor’s Degree Computer Science / Information Technology - Minimum 2 year(s) of working experience as Data Engineer / Data Scientist - Experience in creating and implementing data warehouse using SQL Server SSIS, SSAS, SSRS; able to create dashboard and report using Power BI (for Data Engineering position) -...

  • Python Engineer

    3 months ago


    Jakarta, Indonesia PureLogics Full time

    Python Engineer - Experience required: 2-3 Years - Skills Required: Must have experience in one of the frameworks of Python like Django, Flask or Pyramid whereas Django is highly preferable, and familiar with deployments and AWS "Waspada terhadap Modus Penipuan pada saat proses interview. Perusahaan tidak akan memungut biaya apapun dalam melakukan proses...

  • Data Engineer

    5 months ago


    Jakarta, Indonesia sribu.com Full time

    The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. **Qualification**: -...

  • Data Architect

    6 days ago


    Jakarta, Jakarta, Indonesia IlmuOne Data Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at IlmuOne Data. As a Data Engineer, you will play a critical role in designing, building, and maintaining our data infrastructure.Key ResponsibilitiesData Infrastructure Development: Design, build, and maintain the infrastructure required for optimal extraction, transformation, and...


  • Jakarta, Indonesia Aera Technology & Consulting Full time

    **Responsibilities**: - Write effective Python code to tackle complex issues, but also use your business sense and analytical abilities to glean valuable insights from public databases - Communicate clearly with researchers and help the organization in realizing its objectives - Clearly express the reasoning and logic when writing code in Jupyter notebooks,...

  • Data Architect

    6 days ago


    Jakarta, Jakarta, Indonesia IlmuOne Data Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at IlmuOne Data. As a Data Engineer, you will play a critical role in designing, building, and maintaining the infrastructure required for optimal data extraction, transformation, and loading from various data sources.Key ResponsibilitiesData Infrastructure Development: Design,...


  • Jakarta, Indonesia Permana Solutions Full time

    Craft great code, empathize with our users, review pull requests, share your knowledge, and learn together with other team members - Build, deploy and monitor scalable, stable Python services - Develop and maintain robust data pipelines to collect, manipulate, and analyze large datasets. - Implement and optimize machine learning algorithms in Python. -...

  • Cloud Data Engineer

    3 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    We are seeking a skilled Cloud Data Engineer to join our team. In this role, you will be responsible for designing, building, and maintaining cloud-based data infrastructure and services that enable our organization to process, store, and analyze large volumes of data. You will work closely with data scientists, data analysts, and other stakeholders to...

  • Cloud Data Engineer

    3 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    We are seeking a skilled Cloud Data Engineer to join our team. In this role, you will be responsible for designing, building, and maintaining cloud-based data infrastructure and services that enable our organization to process, store, and analyze large volumes of data. You will work closely with data scientists, data analysts, and other stakeholders to...


  • Jakarta, Indonesia Autobahn Security GmbH Full time

    About Autobahn Security Autobahn Security is a technology start-up with a vision to make cybersecurity and vulnerability management accessible to all businesses. Our SaaS product has been developed by some of the world’s finest cybersecurity researchers and experts. Our SaaS tool allows both cybersecurity experts and non-experts to embark on a journey to...

  • Python Test Engineer

    3 months ago


    Jakarta, Indonesia ATI Business Group Full time

    We are excited to invite a talented Python Test Engineer to join our dynamic team and contribute to our mission of delivering top-notch software solutions. As a Python Test Engineer, you will play a crucial role in ensuring the quality and reliability of our software through the implementation of various automated testing methodologies. If you have a passion...

  • Data Engineer

    5 months ago


    Jakarta, Indonesia PT IONPay Networks (NICEPAY) Full time

    Perform data exploration, data cleaning, data imputation, and feature engineering on unstructured and structured data Designing, developing, and maintaining Data Warehouse Responsible for building and maintaining our end-to-end data pipelines from various data sources Develop or Maintaining ETL Process from multiple sources Analyze and Organize raw...

  • Python Test Engineer

    3 months ago


    Jakarta, Indonesia ATI Business Group Full time

    ATI Business Group is a Global Business Process Management, Technology and Services Company. Our Clients’ businesses are on remarkable journeys as they strive to deliver the best possible outcomes for their customers. Our aim at ATI in working with our Clients’, is to support their continued growth by providing cost effective technology and talented and...

  • Data Engineer

    3 months ago


    Jakarta, Indonesia PT. Prima Fajar Cahaya Surya Full time

    Develop and manage data pipeline & data cleansing - Monitoring and improvement data pipepine - Service request fulfillment & troubleshooting incident or case - Documentation - Ensure complience with regulatory requirements Qualifications / Experience - Bachelor's degree in Information Technology, Computer Science, Information Systems majors. - 1-3 years...