Data Engineer

The Data Engineer role is responsible for designing, implementing, optimising, and managing the data infrastructure framework with an emphasis on security and reliability, and meet the needs of reporting solutions that drive decision-making and improve performance across the business. This role will contribute to the development of Aurecon’s Data Engineering standards and patterns as well as Data Governance framework and standards.

Key responsibilities:

• Develop data models and pipelines to meet the needs of reporting solutions that drive decision making and improve performance across all departments of the business.
• Responsible for the data modelling, profiling and pipeline development of the Analytical data asset
• ·Ensuring data practices integrate into overall data architectures and data management principles (e.g. data governance, data security, metadata, data quality)
• Collect, aggregate, analyse and cleanse structured/unstructured data from multiple sources
• Advising on data architecture, creating data models, delivering data migration, integration and pipelines
• Troubleshooting and solving complex technical problems on large datasets
• Contribute to the development of Aurecon’s Data Engineering standards and patterns
• Contribute to the development of Aurecon’s Data Governance framework and standards
• Designing, implementing, optimising and managing the data infrastructure framework on Azure with an emphasis on security and reliability.
• Managing data infrastructure in a CI/CD (Continuous Integration/Continuous Development) pipeline with appropriate controls and documentation.
• Liaise with Reporting analysts and data analysts and other developers to ensure all pipelines are supported by the correct architectural implementation.
• Work closely with Data Analysts, reporting analysts and domain architects to develop the Data & Analytics framework and Enterprise Data Lakehouse solution
• Comply with the procedures and processes used within the Data & Analytics team, ensuring all work undertaken is executed consistently and professionally.
• Contribute to the triage, analysis and resolution of identified issues and service requests for the Data Lakehouse and Enterprise Reports

Essential Skills and Experience:

• Able to work at Senior-level capacity
• Stakeholder management, project delivery skills as well as clear idea articulation.
• Strong analytical thinking and problem-solving skills with thought leadership and commercial awareness.
• Excellent communications skills required (verbal and written in English)
• Experience in Shared services / IT Consulting services industry is a plus

Qualifications:

• Minimum of 3-5 years experience as a Data Engineer, preferably in an Microsoft Azure environment
• Tertiary qualifications - Bachelor in computer science, software development, information technology, or a related field
• Strong knowledge of Azure, Data Factory or Synapse Pipelines & PowerBI
• Azure Certification / Azure Developer Associate is required
• A wide range of languages, including (but not limited to) SQL, Spark, Python
• Strong capabilities such as performance optimisation (data models, production orchestration and automation) to improve data engineer productivity
• Dimensional Data Modelling experience
• Knowledge and experience in designing, implementing, deploying and configuring Azure resources using Azure DevOps tools and scripting languages
• Data Lakehouse & Enterprise Data Warehouse methodologies
• Understanding of GIT and CI/CD workflow using Azure DevOps
• Support multiple environments (development, test, production) and a strong understanding of SDLC concepts.
• Experience participating in scrum-based agile teams