Advanced Data Solutions Specialist

At the forefront of data engineering, we are seeking a highly skilled professional to drive scalable tech solutions.

This role entails collaborating with architects, analysts, and stakeholders to deliver robust data solutions, focusing on working with Australian clients and the broader APAC region.

1. Data Pipeline Design: Design, build, and maintain scalable data pipelines using Microsoft Fabric, Azure Data Factory, and related Azure services for data ingestion, transformation, and curation.
2. Collaboration: Collaborate with cross-functional teams to deliver high-quality data solutions.
3. Solution Design: Participate in solution design and technical discussions to ensure scalable and secure data workflows.
4. Performance Optimization: Implement performance-optimized and secure data workflows ensuring compliance with governance and quality standards.
5. Documentation: Create and maintain clear, accurate, and up-to-date technical documentation for data assets, pipelines, and processes.
6. Communication: Engage effectively with technical and non-technical team members to ensure shared understanding of requirements and delivery expectations.
7. Tech Solutions: Translate business and analytical requirements into technical solutions leveraging Microsoft Fabric components and Azure services.
8. Environment Monitoring: Ensure a stable development environment following CI/CD best practices monitoring pipeline performance and addressing issues promptly.

1. Experience: Proven experience of more than 6 years in data engineering, data warehousing, and analytics projects covering data ingestion, transformation, curation, and delivery.
2. Knowledge: Strong knowledge of modern data file formats such as Delta Tables, Apache Iceberg, and Parquet.
3. Skills: Proficiency in Python, Spark, Azure Data Factory, and other relevant data transformation tools and frameworks.
4. Data Modelling: Expertise in data modelling across conceptual, logical, and physical layers with knowledge of Dimensional Modelling and Data Vault 2.0 methodologies.
5. Cloud Experience: Experience with cloud data services preferably within the Microsoft ecosystem with exposure to AWS or GCP as an advantage.
6. Hands-on Experience: Hands-on experience with Microsoft Fabric, Databricks, and/or Snowflake.
7. Pipeline Orchestration: Skilled in data pipeline orchestration using Azure Data Factory and related frameworks.
8. CI/CD: Experience with CI/CD pipelines and DevOps practices for automated deployment and integration workflows.
9. IaC: Proficiency in Infrastructure as Code IaC tools such as Terraform or Bicep.
10. Security: Solid understanding of data governance data management practices and security considerations for cloud data platforms.
11. Database Tuning: Competence in database performance tuning optimization and advanced SQL development.
12. Design Patterns: Knowledge of design patterns clean architecture and clean coding principles for scalable data solutions.
13. Testing Methodologies: Familiarity with testing methodologies in data engineering including unit integration and end-to-end testing.

1. Mentorship: Ability to mentor and share knowledge with junior team members.
2. Consulting Mindset: Adaptability in diverse situations and managing changing priorities.
3. Communication: Strong skills in communicating clearly with team members and clients.
4. Non-Functional Requirements: Understanding of non-functional requirements including performance security and data privacy.
5. Passion for Data: Enthusiasm for working with data coding and problem-solving.
6. Innovative Spirit: Drive to continuously improve and innovate pipelines and dashboards.
7. AI Enthusiast: Interest in AI technologies such as Generative AI for self-augmentation and performance improvement.