Data Engineer

Position Accountability Statement
We are seeking a skilled Data Engineer with a strong foundation in data engineering, DevOps practices, data governance, and data warehousing. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines, optimizing data workflows, implementing best practices for data governance, and leveraging DevOps practices to ensure the efficiency and reliability of data operations. This role requires a deep understanding of data engineering concepts, cloud platforms, and infrastructure automation.

Data Warehouse Architecture & Design

• Design, implement, and manage the architecture and infrastructure of the data warehouse, ensuring that it meets business and technical requirements.
• Develop and maintain scalable data models, ensuring that the data warehouse is optimized for query performance and data retrieval.
• Create and manage data schemas, ensuring they align with industry best practices and business needs.
• Design the data warehouse for optimal scalability, ensuring it can handle large volumes of data and high query loads.
• Develop strategies for optimizing query performance and managing large datasets (e.g., partitioning, indexing, caching).
• Optimize the performance of the data warehouse by refining database structures, indexing strategies, partitioning, and optimizing queries.
• Troubleshoot performance bottlenecks, both in the data warehouse and within the ETL processes, and resolve issues to ensure smooth operations.

ETL Development & Data Integration

• Design, implement, and optimize ETL (Extract, Transform, Load) pipelines to move and transform data from various sources into the data warehouse.
• Ensure data integrity, accuracy, and consistency during the ETL process
• Integrate data from multiple sources (databases, APIs, flat files, etc.) into the data warehouse.
• Develop processes to handle data from structured, semi-structured, and unstructured sources.
• Implement Continuous Integration (CI) and Continuous Deployment (CD) practices

specifically for data pipelines. This ensures that data workflows are automatically tested, deployed, and updated with minimal manual intervention.

• Ensure that data pipelines and processing systems are continuously monitored for failures, performance degradation, or anomalies.
• Implement and monitor data quality controls to ensure that data in the warehouse is accurate, consistent, and clean.
• Develop automated validation checks to ensure data integrity and perform regular audits on data accuracy.

Data Governance

• Design and implement systems to control who has access to different datasets, enforcing security measures such as encryption and role-based access control (RBAC).
• Apply data masking techniques or anonymize data to comply with privacy regulations such as GDPR or HIPAA.
• Implement and enforce policies that classify data based on sensitivity or usage, such as personal data, financial data, or confidential business data.
• Track and document the data's journey through various systems, showing where it comes from, how it's transformed, and where it's stored.

Collaboration With Stakeholders

• Work closely with data analysts, business intelligence teams, and data scientists to understand their data needs and provide high-quality data from the warehouse for reporting and analysis.
• Collaborate with IT teams to ensure that the data warehouse infrastructure aligns with overall company architecture and cloud platform

Data Analysis Support

• Oversee and guide the analysis of complex datasets to extract meaningful insights for strategic decision-making.
• Review and validate analytical reports and outputs from the team to ensure high-quality deliverables.
• Analyze large and complex datasets to uncover trends, patterns, and business opportunities.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field.
• 3 to 5 years of experience in data engineering, data management, or similar roles with a focus on data warehousing.
• Strong experience with data engineering concepts and best practices.
• Hands-on experience with designing and implementing large-scale data systems, building and optimizing data pipelines, and working in cloud environments is highly valued.
• Strong proficiency in SQL, PL/SQL, T-SQL and similar technologies.
• Proficient in cloud platforms and related data technologies (Oracle Cloud Infrastructure would be a big plus).
• Hands-on experience with data warehousing solutions and ETL/ELT processes (Oracle

Data Integrator would be a big plus).

• Strong knowledge of data governance principles, including data quality, privacy, and compliance.
• Familiarity with Data Warehouses and data lakes is highly desirable (e.g., Redshift, Snowflake, Synapse, BigQuery, SAP Hana, Db2, s3, ADLS gen2, BigLake)
• Familiarity with DevOps practices, including CI/CD, infrastructure automation, Docker, and Kubernetes is a plus.
• Familiarity with Python, PySpark, Pandas and other relevant libraries is a plus.
• Ability to communicate technical concepts to non-technical stakeholders and document solutions clearly.
• Ability to work closely with data scientists, analysts, and business stakeholders to understand data requirements and optimize solutions.
• Strong analytical and troubleshooting skills to handle complex data issues and improve data flow efficiency.
• In-depth knowledge in MS EXCEL/Google Shee
• Strong experience with data visualization tools (e.g., Oracle Analytics, Sisense, Power BI).