Data Engineer

What You'll Be Doing:

• Design and implement data processing systems using Microsoft Fabric, Azure Data Analytics, Databricks and other distributed frameworks like Hadoop, Spark, Snowflake, Airflow, or other similar technologies.
• Build data pipelines to ingest data from various sources such as databases, APIs, or streaming platforms.
• Integrate and transform data to ensure its compatibility with the target data model or format.
• Design and optimize data storage architectures, with One lake, data lakes, data warehouses, Serverless and any distributed file systems.
• Implement techniques like partitioning, compression, or indexing to optimize data storage and retrieval.
• Identify and resolve bottlenecks, tuning queries, and implementing caching strategies to enhance data retrieval speed and overall system efficiency.
• Design and implement data models that support efficient data storage, retrieval, and analysis.
• Collaborate with data scientists and analysts to understand their requirements and provide them with well-structured and optimized data for analysis and modeling purposes.
• Utilize frameworks like Spark to perform distributed computing tasks, such as parallel processing, distributed data processing, or machine learning algorithms
• Implement security measures to protect sensitive data and ensuring compliance with data privacy regulations.
• Establish data governance practices to maintain data integrity, quality, and consistency.
• Monitor system performance, identify anomalies, and conduct root cause analysis to ensure smooth and uninterrupted data operations.
• Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand their requirements and provide technical solutions.
• Communicating complex technical concepts to non-technical stakeholders in a clear and concise manner.
• Ability to work under Agile and Scrum development methodologies
• Stay updated with emerging technologies, tools, and techniques in the field of big data engineering. Exploring and recommending new technologies to enhance data processing, storage, and analysis capabilities.
• Train and mentor junior data engineers, providing guidance and knowledge transfer.

What We're Looking For:

• Strong analytical thinking and problem-solving skills
• Strong communication skillset – ability to translate technical details to business/non-technical stakeholders
• Extensive experience in designing and building data pipelines (ELT/ETL) for large-scale datasets.
• Familiarity with tools like Databricks, Apache Nifi, Apache Airflow, or Informatica is advantageous
• Proficiency in programming languages such as Python, R or Scala is essential.
• In-depth knowledge and experience with distributed systems and technologies, including Apache Hadoop, Spark, Hive or similar frameworks.
• Familiarity with cloud-based platforms like AWS, Azure, or Google Cloud is highly desirable.
• Solid understanding of data processing techniques such as batch processing, real-time streaming, and data integration.
• Experience with data analytics tools and frameworks like Apache Kafka, Apache Flink, or Apache Storm is a plus.
• Experience with Azure Data Services - Databricks and Data Factory
• Experience with Git repository maintenance and DevOps concepts
• Familiarity with building, testing, and deploying process
• Additional certifications in big data technologies or cloud platforms are advantageous

Missing one or two of these qualifications? We still want to hear from you If you bring a positive mindset, we'll provide an environment where you feel valued and empowered to learn and grow.

Offer

• Stable employment. On the market since 2008, 1500+ talents currently on board in 7 global sites.
• "Office as an option" model. You can choose to work remotely or in the office.
• Flexibility regarding working hours and your preferred form of contract.
• Comprehensive online onboarding program with a "Buddy" from day 1.
• Cooperation with top-tier engineers and experts.
• Unlimited access to the Udemy learning platform from day 1.
• Certificate training programs. Lingarians earn 500+ technology certificates yearly.
• Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.
• Internal Gallup Certified Strengths Coach to support your growth.
• Grow as we grow as a company. 76% of our managers are internal promotions.
• A diverse, inclusive, and values-driven community.
• Autonomy to choose the way you work. We trust your ideas.
• Create our community together. Refer your friends to receive bonuses.
• Activities to support your well-being and health.
• Plenty of opportunities to donate to charities and support the environment.
• Great Place to Work Certified Employer in the Philippines