Data Engineer

Job Role: Data Engineer

Employment Type: Full Time

Work Location: Makati

Work Arrangement: Hybrid (2x a week onsite)

Key Responsibilities:

Create proof-of-concepts and implement best-fit data/platform solutions aligned to the business/product owner's initiatives

Promote and define enterprise-grade data engineering standards, including coding practices, testing frameworks, version control, and CI/CD processes

Design, build, support complete data lifecycle – from ingestion, transformation, storage, access – ensuring data integrity, security, and quality

Collaborate closely with data scientists, data analysts, data product owners, third-party teams, and business stakeholders to translate needs into data-driven/technical requirements and solutions

Ensure alignment with regulatory requirements, data governance frameworks, and internal data policies, fostering trust and accountability in data assets

Keep updated of emerging data technologies and tools, evaluating and integrating them where they can deliver strategic advantage

Identify opportunities for innovation in data infrastructure and automation

Implement observability practices to monitor data platforms' performance and reliability, practicing continuous improvement where necessary

Coach and mentor junior to mid-level data engineers, creating high-performing engineering culture

Qualifications:

Experience

years of experience in data engineering

Proven and validated experience in designing and building large-scale data platforms or pipelines (batch, real-time) with varying security/governance compliance requirements from different business entities/data sources

Understanding of data architectures (data lake, data warehouse, lakehouse, delta architecture)

Hands-on experience with various Cloud platform services to support data engineering and data lifecycle management

Proficiency in data manipulation and pipeline development with scripting background

Working knowledge of data cataloging, lineage tracking and access controls with familiarity on regulatory requirements and data privacy/security best practices

Comfortable in designing, implementing, and managing CI/CD pipelines (for data pipelines), version control, Infrastructure as Code with testing and monitoring frameworks

Experience implementing data quality, observability, and pipeline monitoring

Prior experience in a lead and architect capacity, including technical leadership and decision making

Skills and Competencies

Essential

o   Design, implementation, maintenance, and support of organization-wide data solutions and end-to-end data pipelines

o   Expertise in data modeling, normalization, and schema design

o   Enablement and operationalization of cloud components in relation to the data solutions

o   Ingestion and integration of structured, semi-structured, and unstructured data from diverse data sources

o   Design and implementation of CI/CD solutions for data pipelines

o   Understanding of data access controls, lineage, and privacy regulations

o   Establish, enable, and implement governance, compliance requirements, security of data and data pipelines via the Data Platform

o   Implement pipeline health checks, monitoring, and alerting

o   Cost and resource utilization optimization of pipelines, data solutions

o   Ability to align data/data platform initiatives with data productization and long-term organization goals

o   Strong written and verbal communication to business proponents around design and implementation of data solutions and end-to-end data pipelines

o   Encourage innovation and continuous improvement in data platform/data delivery

o   Proficiency on various Data Engineering Tools:

§  Platform – Databricks

§  Programming - Python, SQL, Shell Scripting

§  Orchestration – Apache AirFlow

§  Processing – Apache Spark, Kafka

§  Storage – S3, NoSQL DB

§  Cloud – AWS

§  Integration – AWS Integration Services, APIs, CDCs

§  CI/CD – GitHub, GitHub Actions, Databricks Asset Bundles, IaC (Terraform)

Desirable

o   Knowledge on Azure and GCP data engineering-related services

o   Skilled in managing multiple SIs around data delivery

o   Has other relevant Data, DevOps, and Cloud certifications

o   Data Engineering Tools:

§  Platform - Microsoft Fabric

§  Cloud - GCP, Azure

§  Integration - AWS Integration Services, various iPaaS

Education / Certifications:

Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field

Databricks Fundamentals Certification

AWS Cloud Practitioner Certification

Other related certifications are a plus