Senior Data Engineer (Contract)

Sydney

6 Months (Initially)

$970pd Inc Super

About Us:

Leading Insurance business in Australia are seeking a highly skilled Senior Data Engineer to join their Sydney team on a 6-month contract basis.

Key Responsibilities:

Data Architecture: Design and implement a robust data lakehouse for diverse data storage and analytics, ensuring optimal performance and scalability.

Data Pipelines: Implement efficient data pipelines to ingest and process data into the data lakehouse, enabling timely and accurate insights generation.

Terraform Scripts: Develop Terraform scripts for application infrastructure provisioning, automating deployment tasks and enhancing efficiency.

Scalability Planning: Plan for capacity and scale up infrastructure as needed, anticipating future growth and ensuring seamless scalability of the platform.

CSIO Team Collaboration: Collaborate with the CSIO team to ensure cybersecurity and secure design principles are integrated into the GenAI platform, mitigating risks and ensuring compliance.

Requirements:

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in Azure Databricks
• Strong expertise in designing and implementing data architectures, including data lakehouses.
• Experience in developing and maintaining data pipelines for large-scale data processing.
• Hands-on experience with Azure DevOps for CI/CD implementation.
• Proficiency in scripting languages such as Python (Pyspark)
• Experience in monitoring and maintaining AI model performance (Desirable)

If you feel you possess the relevant skills to the above, apply now with most updated CV!