Big Data Engineer

3 weeks ago

Manila, National Capital Region, Philippines Activefence Full time
Description

We are seeking a Senior Big Data Engineer with a strong background in managing structured and unstructured data pipelines, who thrives in a fast-paced AI-focused environment. You will be instrumental in building and scaling our data lake architecture, supporting a system designed to fuel intelligent AI agents for data collection, labeling, and analytical reasoning. This includes integrating vector databases and optimizing for retrieval-augmented generation (RAG) workflows deployed on AWS Bedrock and other AI stacks.

Responsibilities
  • Design and implement scalable ingestion pipelines for structured/unstructured data using AWS and Databricks Unity Catalog.
  • Build and maintain high-throughput ETL/ELT pipelines with Apache Airflow and Databricks.
  • Architect and manage data modeling, storage, and indexing strategies in PostgreSQL and RDS, ensuring compatibility with AI retrieval systems.
  • Integrate and manage vector databases to support fast semantic and embedding-based search in RAG pipelines.
  • Collaborate with AI engineers to ensure seamless compatibility with LangGraph and LangSmith agent systems.
  • Implement robust data validation, lineage, and governance systems using Unity Catalog.
  • Optimize performance across distributed compute environments (Databricks, EC2).
  • Deploy and maintain Lambda-based microservices for scalable, real-time data ingestion and enrichment.
Requirements
  • 5+ years working with big data systems in production environments.
  • Proven expertise with Databricks, Unity Catalog, and Apache Spark.
  • Proficiency in Airflow, AWS stack (Lambda, EC2, RDS), and cloud-based data lake architectures.
  • Strong SQL and database design skills (PostgreSQL preferred).
Preferred Qualifications
  • Experience with AI agent pipelines or large-scale ML model support.
  • Working knowledge of vector databases (Chroma, Pinecone, FAISS).
  • Solid understanding of data lifecycle management in ML/AI contexts.
  • Bonus: Familiarity with LangGraph, LangSmith, LangChain, or similar agent orchestration tools.
  • Emphasis on data observability, security, and lineage tracking.
  • Hands-on with RAG architecture, including vector storage and semantic retrieval.
  • Exposure to AWS Bedrock and model deployment orchestration.
About ActiveFence

ActiveFence is the leading provider of security and safety solutions for online experiences, safeguarding more than 3 billion users, top foundation models, and the world's largest enterprises and tech platforms every day.

As a trusted ally to major technology firms and Fortune 500 brands that build user-generated and GenAI products, ActiveFence empowers security, AI, and policy teams with low-latency Real-Time Guardrails and a continuous Red Teaming program that pressure-tests systems with adversarial prompts and emerging threat techniques. Powered by deep threat intelligence, unmatched harmful-content detection, and coverage of 117+ languages, ActiveFence enables organizations to deliver engaging and trustworthy experiences at global scale while operating safely and responsibly across all threat landscapes.


#J-18808-Ljbffr

  • Data Engineer

    4 days ago

    Manila, National Capital Region, Philippines Tenet Global Business Center, Inc. Full time ₱1,200,000 - ₱2,400,000 per year

    Work Arrangement:Night shift work scheduleHybrid work setupBenefits:15% Night differential20 Paid Time Off (PTO) per yearAnnual AppraisalAnnual IncentiveHybrid Work ArrangementHMO with FREE dependentsGroup life insuranceThe Data Engineer is primarily responsible for participating in the design, development, and implementation of solutions for the data...

  • Data Engineer

    4 days ago

    Manila, National Capital Region, Philippines Unison Group Full time ₱1,200,000 - ₱1,500,000 per year

    Role OverviewWe are looking for a skilled Data Engineer with 5 years of hands-on experience in designing, developing, and optimizing big data pipelines and solutions. The ideal candidate will have strong expertise in SQL, Python, Apache Spark, Hive, and Hadoop ecosystems and will be responsible for building scalable data platforms to support business...

  • Data Engineer

    4 days ago

    Manila, National Capital Region, Philippines Unison Group Full time ₱900,000 - ₱1,200,000 per year

    Role OverviewWe are looking for a skilled Data Engineer with 5 years of hands-on experience in designing, developing, and optimizing big data pipelines and solutions. The ideal candidate will have strong expertise in SQL, Python, Apache Spark, Hive, and Hadoop ecosystems and will be responsible for building scalable data platforms to support business...

  • Data Engineer

    4 days ago

    Manila, National Capital Region, Philippines InvestEd Full time ₱1,200,000 - ₱3,600,000 per year

    We are looking for a Data Engineer to develop, optimize, and maintain our data infrastructure. In this role, you will be responsible for building and optimizing data pipelines, ensuring cost and compute efficiency, and enabling high-quality, curated datasets for analytics and data science. This role requires strong data engineering and problem-solving...

  • Data Engineer

    4 days ago

    Manila, National Capital Region, Philippines Unison Group Full time ₱900,000 - ₱1,200,000 per year

    We are seeking a skilled Data Engineer to join our team in Singapore. The ideal candidate will have strong expertise in SQL, Python, Spark, and Hive, with hands-on experience building scalable data pipelines, data models, and analytical solutions. You will work closely with data scientists, analysts, and business stakeholders to deliver high-quality data...

  • Data Engineer

    4 days ago

    Manila, National Capital Region, Philippines Unison Group Full time ₱900,000 - ₱1,200,000 per year

    We are seeking a skilled Data Engineer to join our team in Singapore. The ideal candidate will have strong expertise in SQL, Python, Spark, and Hive, with hands-on experience building scalable data pipelines, data models, and analytical solutions. You will work closely with data scientists, analysts, and business stakeholders to deliver high-quality data...

  • Data Engineer

    4 days ago

    Manila, National Capital Region, Philippines Fujitsu Full time ₱600,000 - ₱1,200,000 per year

    Site: Manila and CebuExperience inDatabricks, Python, PySpark, Azure, SQL, ETLfor additional skills/certifications.big plusif the applicant has these:Microsoft Fabric Data Engineer Associate (DP-700)Databricks LakehouseDatabricks FundamentalsWhat you'll doConsulting: Define strategy and solutions with clients.Azure Integration: Manage data integration using...

  • Data Engineer

    4 days ago

    Manila, National Capital Region, Philippines YONDU INC. Full time ₱600,000 - ₱1,200,000 per year

    The Data Engineer will be responsible for designing, building, and maintaining robust data infrastructure and pipelines that support current and future business needs. This includes transforming raw data into actionable insights through scalable and secure pipelines, managing data assets effectively, and enabling seamless access to data for analytical and...

  • Data Engineer

    2 days ago

    Manila, National Capital Region, Philippines SiteMinder Full time ₱1,200,000 - ₱2,400,000 per year

    At SiteMinder we believe the individual contributions of our employees are what drive our success. That's why we hire and encourage diverse teams that include and respect a variety of voices, identities, backgrounds, experiences and perspectives. Our diverse and inclusive culture enables our employees to bring their unique selves to work and be proud of...

  • Data Engineer

    2 days ago

    Manila, National Capital Region, Philippines SiteMinder Full time ₱144,000 - ₱600,000 per year

    At SiteMinder we believe the individual contributions of our employees are what drive our success. That's why we hire and encourage diverse teams that include and respect a variety of voices, identities, backgrounds, experiences and perspectives. Our diverse and inclusive culture enables our employees to bring their unique selves to work and be proud of...