Bluefin Resources is proud to partner with a prestigious government agency to recruit a Senior Data Engineer for a long-term contract, with strong potential for extension. This is an exciting opportunity to contribute to a multi-year data modernisation initiative, leveraging the Azure Data Platform and technologies such as Databricks.

Primary purpose of the role

The role builds the data pipeline by integrating various data sources and the maintenance, improvement, cleaning, and manipulation of data from divisional and other databases.

Key accountabilities

• Create and maintain the implementation of an optimal data pipelines from a wide variety of internal and external sources into a large cloud environment and work with genuinely large datasets and implement against reference designs to meet the functional requirements of the cluster, branch and consumers.
• Identify, design, and implement internal process improvements: automating manual processes, streamlining data delivery and re-designing infrastructure for greater scalability and elegance.
• Build and support the infrastructure as code required for the ELT (extract, load, transform) of data into a data lake including SQL and a range of native ‘big data’ cloud technologies.
• Keep the virtualised separation of cloud environments and play a strong role in contributing to protecting the data inside a secured perimeter.
• Build and support the ETL (extract, transform, load) of data including data cleansing into a data warehouse and create of data tools and scripts for Analytics, Business Intelligence and Data Science teams to assist in building highly optimised products ready for a production environment.
• Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement and build processes supporting data transformation, data structures, metadata, dependency and workload management.
• Actively apply expertise in SQL and provide support in RDBMS design, data flow and analysis activities, and troubleshoot data issues and present / implement the solution.
• Ensure that Data Governance is embedded at each point of the data management lifecycle and is a key consideration of all parts of any implementation or data service and develop process to automate manual data processes, optimising data delivery, real-time data processes for greater scalability and business value.

Key Knowledge and Experience

• Proficiency in Azure Cloud data platform with exceptional skills Databricks, ADF.
• Hands on experience programming languages like Python and SQL, database management, cloud computing, data warehousing, and big data technologies, along with strong problem-solving and collaboration abilities.
• Proficiency in Data Engineering practice including DataOps, version control and DevOps process management.
• Effectively conveying technical information to both technical and non-technical audiences.
• Working effectively with cross-functional teams, including data analysts, and other engineers and Business Analytics.

What's Available:

This is an initial 6 month contract with a strong possibility of extension. The role offers flexible working options and is based in Sydney.

You'll be working alongside a team of dedicated professionals, contributing to a meaningful and high-impact project where your expertise will make a real difference.

To apply, please submit your updated resume online. For a more in-depth discussion, feel free to reach out to Lee Bartlett at 0410 744 438.