🧠 Job Title: Lead Data Engineer - PySpark & Data Ingestion
📍 Location: Sydney, NSW, Australia
💰 Salary: $190,000 + Super
🏦 Industry: Financial Services
🕹️ Type: Full-time | Hybrid Working Environment

Your Impact 🌟

We're looking for a skilled Lead Data Engineer to join our Information Security team and play a key role in delivering and supporting our Security Insights platform powered by Palantir. You'll be at the heart of a critical function-designing, implementing, and operating data pipelines that secure and strengthen our organisation's future. 🔐💡

What You'll Be Doing 🔧

• 💻 Transform unstructured datasets into consumable formats using PySpark/Python or Apache Spark (Java/Scala considered too).
• 📥 Ingest data from diverse sources (REST APIs, SFTP, Bash, SQL) while managing network connectivity, firewalls, and proxies in Linux environments.
• ⚙️ Design and optimise data pipelines in distributed systems, using smart techniques like pagination, partitioning, and broadcasting.
• 🧪 Improve operations and CI/CD processes while working within Agile/Scrum and tools like Jira and ServiceNow.
• 🤝 Act as a Tech Lead / 2IC, owning stakeholder engagement and promoting strong data governance and management practices.

What You'll Bring 💼

• ✅ Deep hands-on experience with PySpark, Python, and distributed data systems.
• 🔐 Strong background working with data ingestion pipelines and security-aware architectures.
• 🌐 Confidence working in Linux environments with system-level integrations.
• 🧰 Experience with ITSM, CI/CD pipelines, and Agile delivery practices.
• 📊 Passion for data quality, ownership, and stakeholder engagement.

Nice to Have (But Not a Dealbreaker!) 🎯

• 🛡️ Experience with data governance, access control, and master data management.
• 🧱 Familiarity with multi-tenant data lake platforms like Databricks, Snowflake, or Cloudera.

Why You'll Love It Here ❤️

• 🏛️ Be part of a 200+ year legacy, focused on building better futures.
• 📈 Grow your career in a secure, innovative, and supportive environment.
• 👨‍👩‍👧‍👦 Enjoy flexible working, paid parental leave, and ongoing super contributions during extended leave.
• 💬 Join a purpose-driven team with a strong can-do culture and endless learning opportunities.

Ready to turn big data into big impact? Apply now and help shape the future of secure, data-driven decision-making! 🚀📊🔐