Data Engineer with Python

7 months ago


Jakarta, Indonesia Unison Consulting Pte Ltd Full time

Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform.
- Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale.
- Optimize and troubleshoot AWS Glue jobs for performance and reliability.
- Utilize Python and PySpark to efficiently handle large volumes of data during the ingestion process.
- Design and implement scalable data processing solutions using PySpark to transform raw data into a structured and usable format.
- Create and maintain ETL processes using Python and PySpark to move and transform data between different systems.
- Optimize ETL workflows for performance and efficiency.
- Collaborate with data architects to design and implement data models that support business requirements.
- Ensure data structures are optimized for analytics and reporting.
- Work with distributed computing frameworks, such as Apache Spark, to process and analyze large-scale datasets.
- Manage and optimize databases, both SQL and NoSQL, to support data storage and retrieval needs.
- Implement indexing, partitioning, and other database optimization techniques.
- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver effective solutions.
- Implement monitoring solutions to track data pipeline performance and proactively identify and address issues
- Ensure compliance with data privacy regulations and company policies.
- Stay abreast of industry trends and advancements in data engineering, Python, and PySpark.

**Requirements**:

- Proficiency in Python and PySpark.
- Strong knowledge of data engineering concepts and best practices.
- Hands-on experience with AWS Glue and other AWS services.
- Experience with big data technologies and distributed computing.
- Familiarity with database management systems (SQL and NoSQL).
- Understanding of ETL processes and data modeling.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Bachelor's degree in Computer Science, Information Technology, or a related field.


  • Data Engineer

    6 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    Research, build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using JavaScript or other languages - Contributes to team effort by accomplishing related results as needed. - Explore/learn new technologies that can complement or replace our current stack to improve it. -...

  • Data Engineer

    6 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    Job Description:Research, build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using JavaScript or other languagesContributes to team effort by accomplishing related results as needed.Explore/learn new technologies that can complement or replace our current stack to...


  • Jakarta, Jakarta, Indonesia IlmuOne Data Full time

    At IlmuOne Data, we are seeking a highly skilled Data Engineer Expert to join our team. This is an exceptional opportunity for a talented individual to take their career to the next level.About the RoleWe are looking for a motivated and detail-oriented Data Engineer who can design, build, and maintain scalable data infrastructure using JavaScript and other...

  • Python Data Scientist

    6 months ago


    Jakarta, Indonesia Nucleus Health Full time

    **Responsibilities**: - Write effective, high-quality Python code to solve problems - Clearly explain the reasoning behind the code in Jupyter notebooks - Debug code and produce thorough documentation - Using free datasets (such as Kaggle, the UN, the US government, etc.), use your data analytic skills to formulate and respond to important business issues -...

  • IT Data Engineer

    5 months ago


    Jakarta, Indonesia PGI Data Full time

    Develop, construct, test and maintain cloud architectures to nurture data-focused products and internal tools Align said architecture with business requirements Acquire and integrate data from external sources Bachelor’s degree in any major (preferably in Informatics, Computer Science, Software Engineer) 2-4 years experience Good knowledge in Data...

  • Data Engineer

    5 months ago


    Jakarta, Indonesia Group Hijra (Hijra Bank, ALAMI P2PL) Full time

    Hijra (previously known as the ALAMI Group) is a financial technology company that follows sharia principles. It was founded in 2018 by Dima Djani, Harza Sandityo, and Bembi Juniar. The company offers a range of services, including a peer-to-peer lending platform, mobile banking app, and mortgage all of which are based on sharia principles. Hijra has a...


  • Jakarta, Indonesia Aera Technology & Consulting Full time

    **Responsibilities**: - Write effective Python code to tackle complex issues, but also use your business sense and analytical abilities to glean valuable insights from public databases - Communicate clearly with researchers and help the organization in realizing its objectives - Clearly express the reasoning and logic when writing code in Jupyter notebooks,...


  • Jakarta, Indonesia Permana Solutions Full time

    Craft great code, empathize with our users, review pull requests, share your knowledge, and learn together with other team members - Build, deploy and monitor scalable, stable Python services - Develop and maintain robust data pipelines to collect, manipulate, and analyze large datasets. - Implement and optimize machine learning algorithms in Python. -...

  • Cloud Data Engineer

    6 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    We are seeking a skilled Cloud Data Engineer to join our team. In this role, you will be responsible for designing, building, and maintaining cloud-based data infrastructure and services that enable our organization to process, store, and analyze large volumes of data. You will work closely with data scientists, data analysts, and other stakeholders to...

  • Cloud Data Engineer

    6 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    We are seeking a skilled Cloud Data Engineer to join our team. In this role, you will be responsible for designing, building, and maintaining cloud-based data infrastructure and services that enable our organization to process, store, and analyze large volumes of data. You will work closely with data scientists, data analysts, and other stakeholders to...

  • Python Test Engineer

    5 months ago


    Jakarta, Indonesia ATI Business Group Full time

    We are excited to invite a talented Python Test Engineer to join our dynamic team and contribute to our mission of delivering top-notch software solutions. As a Python Test Engineer, you will play a crucial role in ensuring the quality and reliability of our software through the implementation of various automated testing methodologies. If you have a passion...

  • Data Engineer

    2 weeks ago


    Jakarta, Indonesia IlmuOne Data Full time

    **Work Responsibilities**: - Create and maintain an optimal/stable data pipeline - Collect complex data for the functional and non-functional needs of the company & clients - Identify, design, and implement the development process which includes process automation, optimization of data provision, redesign of scalable infrastructure - Develop optimal...


  • Jakarta, Jakarta, Indonesia IlmuOne Data Full time

    Job Title: Senior GIS Data AnalystThe Senior GIS Data Analyst will play a key role in developing and integrating machine learning models with geographic information systems (GIS) for the analysis, visualization, and processing of spatial data. This role requires collaboration with cross-functional teams to deliver data-driven solutions, automate geospatial...

  • Junior Gis Ml Engineer

    2 months ago


    Jakarta, Indonesia IlmuOne Data Full time

    The Junior GIS ML Engineer will assist in developing and integrating machine learning models with geographic information systems (GIS) for the analysis, visualization, and processing of spatial data. This role requires collaboration with cross-functional teams to deliver data-driven solutions, automate geospatial tasks, and provide insights from spatial...

  • Data Architect

    1 month ago


    Jakarta, Jakarta, Indonesia IlmuOne Data Full time

    Job Description:We are seeking a highly skilled Data Architect to join our team at IlmuOne Data. The ideal candidate will have a strong background in designing and implementing data infrastructure solutions using JavaScript and other languages.The successful candidate will be responsible for:Designing and implementing data pipelines using GCP, AWS, and...

  • Python Test Engineer

    5 months ago


    Jakarta, Indonesia ATI Business Group Full time

    ATI Business Group is a Global Business Process Management, Technology and Services Company. Our Clients’ businesses are on remarkable journeys as they strive to deliver the best possible outcomes for their customers. Our aim at ATI in working with our Clients’, is to support their continued growth by providing cost effective technology and talented and...

  • Data Engineer

    5 months ago


    Jakarta, Indonesia PT. Prima Fajar Cahaya Surya Full time

    Develop and manage data pipeline & data cleansing - Monitoring and improvement data pipepine - Service request fulfillment & troubleshooting incident or case - Documentation - Ensure complience with regulatory requirements Qualifications / Experience - Bachelor's degree in Information Technology, Computer Science, Information Systems majors. - 1-3 years...

  • Data Engineer

    2 weeks ago


    Jakarta, Indonesia IlmuOne Data Full time 8,000,000 - 17,000,000

    Work Responsibilities:Create and maintain an optimal/stable data pipelineCollect complex data for the functional and non-functional needs of the company & clientsIdentify, design, and implement the development process which includes process automation, optimization of data provision, redesign of scalable infrastructureDevelop optimal infrastructure for the...


  • Kota Administrasi Jakarta Pusat, Jakarta, Indonesia IlmuOne Data Full time

    Are you a seasoned expert in building scalable data systems? We are seeking a highly skilled Data Systems Architect to join our team at IlmuOne Data.About the RoleIn this role, you will be responsible for designing and implementing robust data infrastructure to support our software applications. You will collaborate closely with cross-functional teams to...

  • Data Scientist

    4 weeks ago


    Jakarta, Indonesia IlmuOne Data Full time

    **Data Scientist Job Description**: **Work responsibilities**: - Assist in methodology research, data processing, and documentation to conduct independent self-guided exploration of research and frame open-ended industry questions, including, but not limited to: - Conducts statistical analyses to develop strategies. - Builds predictive models and...