Data Engineer

3 days ago

Cebu City, Central Visayas, Philippines Codev Insights Full time
Overview

Position SummaryWe are seeking an experienced Data Engineer to specialize in the orchestration and operational management of a new data warehouse project. This role is centered around an expert-level mastery of Google Cloud Composer (Apache Airflow) and the art of DAG (Directed Acyclic Graph) authoring. You will be responsible for scheduling, automating, and monitoring all data pipelines, ensuring they run on time, in the correct order, and with robust error handling. Your expertise will be vital in building a resilient, observable, and highly automated data ecosystemthat the entire team can depend on.

Responsibilities

Key Responsibilities

  • Design, build, and manage the complete orchestration framework for all data pipelinesusing Google Cloud Composer.
  • Author complex, dynamic, and maintainable DAGs in Python, implementingsophisticated dependency management, triggers, and scheduling logic.
  • Serve as the team's subject matter expert on all DAG Authoring/Best Practices,including idempotency, task modularity, backfilling, and performance tuning.
  • Develop a comprehensive monitoring and alerting strategy for all orchestrated pipelinesto proactively identify and resolve issues.
  • Establish and enforce CI/CD processes for deploying and testing DAGs to ensureoperational stability.
  • Collaborate with the ingestion and transformation engineers to understand pipelinedependencies and integrate their workflows into the master orchestration plan.
  • Own the operational health of the data platform, managing retries, handling failures, andcommunicating pipeline status to stakeholders.
Qualifications
  • Core Required SkillsExpert-Level DAG Authoring: Demonstrable mastery of authoring complex DAGs inPython. This must include deep knowledge of Airflow operators and sensors, dynamicDAG generation, managing task dependencies and SLAs, and creating idempotenttasks.
  • Cloud Composer (Apache Airflow): Deep, hands-on experience managing aproduction Cloud Composer environment, including configuration, security, scaling, andtroubleshooting.
  • Python for Orchestration: Expert-level Python proficiency as it applies to writing clean,efficient, and testable Airflow DAGs.
  • CI/CD for Data Pipelines: Experience implementing continuous integration anddeployment workflows for DAGs using tools like Git and Google Cloud Build.
  • Monitoring & Alerting: Strong knowledge of setting up and using monitoring tools (like Google Cloud's operations suite) to track pipeline performance and alert on failures.

#J-18808-Ljbffr

  • Data Engineer

    5 days ago

    Cebu City, Central Visayas, Philippines RAFI MICRO-FINANCE Full time $90,000 - $120,000 per year

    Data EngineerLocation: Cebu (Head Office)Employment Type: Project-BasedAbout the RoleWe are looking for a Data Engineer who will help us design, build, and manage data systems that power our organization's decision-making. You'll work on data pipelines, integration, and automation, ensuring that our data is accurate, secure, and ready to support analytics...

  • Data Engineer

    5 days ago

    Cebu City, Central Visayas, Philippines Cebu Landmasters, Inc. Full time $80,000 - $120,000 per year

    About the role Cebu Landmasters, Inc. is seeking a talented Data Engineer to join our growing team in Cebu City, Cebu. As a full-time Data Engineer, you will play a pivotal role in will in designing, building, and maintaining the data infrastructure to support the data management initiatives. This role focuses on developing robust data repositories, such as...

  • Data Engineer

    3 days ago

    Cebu City, Central Visayas, Philippines Codev Insights Full time

    OverviewFind your new position here at CoDev. We always have a variety of open positions, ranging from software developers to dev ops, and more. Explore our available openings below, or contact us for help in finding the right position for you.We are a fast-growing SaaS company modernizing the school and sports photography industry. Our platform helps...

  • Senior Data Engineer

    5 days ago

    Cebu City, Central Visayas, Philippines Optum Full time $90,000 - $120,000 per year

    Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by diversity and inclusion,...

  • Sr Data Engineer

    5 days ago

    Cebu City, Central Visayas, Philippines Optum, a UnitedHealth Group Company Full time $90,000 - $120,000 per year

    Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by diversity and inclusion,...

  • Senior Data Engineer

    1 week ago

    Cebu City, Central Visayas, Philippines beBeeDataEngineer Full time ₱800,000 - ₱1,200,000

    Job OverviewWe are seeking a highly skilled Data Engineer to join our team. The successful candidate will be responsible for designing, implementing, and maintaining robust data pipelines using Microsoft technologies.Key ResponsibilitiesThe ideal candidate will have deep expertise in ETL development, database engineering, reporting solutions, data...

  • Senior Data Engineer

    5 days ago

    Cebu City, Central Visayas, Philippines Optum Full time $80,000 - $120,000 per year

    Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers,...

  • Data and Reporting Engineer

    1 week ago

    Cebu City, Central Visayas, Philippines QBE Insurance Full time

    OverviewData and Reporting Engineer at QBE Insurance – responsible for collecting, transforming, and storing data from various sources, ensuring data quality and integrity, and creating meaningful reports and dashboards for stakeholders.ResponsibilitiesData Extraction and Transformation: Gather data from diverse sources, such as databases, APIs, and flat...

  • Data and Reporting Engineer

    6 days ago

    Cebu City, Central Visayas, Philippines QBE Europe Full time

    Primary DetailsTime Type: Full timeWorker Type: EmployeeA Data and Reporting Engineer is responsible for collecting, transforming, and storing data from various sources, ensuring data quality and integrity, and creating meaningful reports and dashboards for stakeholders. They bridge the gap between data sources and end-users, enabling data-driven...

  • Senior Big Data Engineer

    7 days ago

    Cebu City, Central Visayas, Philippines beBeeDataEngineering Full time $90,000 - $120,000

    Data Engineering PositionWe are seeking an experienced Data Engineer to join our team. The successful candidate will be responsible for designing, building, and maintaining large-scale data systems, ensuring high performance, scalability, and reliability.The ideal candidate will have hands-on experience with cloud data services, ETL/ELT pipelines, and Agile...