Senior Data Engineer

Contract Type:

Contractor

Location:

Canberra - Australian Capital Territory

Industry:

Information & Communication Technology

Contact Name:

Craig Nel

Contact Email:

craign@coxpurtell.com.au

Contact Phone:

0292203400

Date Published:

13-Feb-2025

 

 

Are you an experienced Data Engineer looking for an opportunity to work on high-impact government projects?

This role offers the chance to develop and optimise data pipelines in Azure, leveraging Python, SQL, and Azure Data Factory and Databricks to drive efficient and scalable data solutions.

This is a 4-month contract role with the possibility of a 12-month extension, based in Canberra with a hybrid work arrangement (minimum 3 days in-office per week). The position requires candidates to hold or be eligible for a Baseline security clearance and allows for a maximum of 40 hours per week.

Role Responsibilities:

  • Develop, optimise, and maintain data pipelines using Python & SQL in Azure Databricks.
  • Design and implement ETL/ELT workflows in Azure Data Factory for efficient data transformation and loading.
  • Apply Kimball dimensional modelling and Medallion architecture best practices for structured, scalable data solutions.
  • Collaborate with business stakeholders to translate requirements into technical data solutions.
  • Implement and manage CI/CD pipelines using Azure DevOps & Git to ensure automated deployments and version control.
  • Monitor, troubleshoot, and enhance Databricks jobs and queries for performance optimisation.
  • Work closely with data analysts and BI teams to deliver well-structured, high-quality datasets.
  • Ensure compliance with data governance, security, and privacy best practices.
  • Contribute to code quality improvement through peer reviews, best practices, and knowledge sharing.

Requirements

Technical Skills & Experience:

  • Proven experience in developing ETL/ELT processes for large-scale data movement and transformation in a cloud environment.
  • Expertise in Python, SQL, and Azure Databricks for data pipeline development.
  • Experience optimising query performance in distributed computing engines (Spark), Azure SQL, Python, or R.
  • Hands-on experience with Azure Data Factory, Azure DevOps, and Git for CI/CD pipelines.
  • Strong understanding of Kimball dimensional modelling (fact/dimension tables, star/snowflake schemas).
  • Knowledge of Medallion architecture for structuring data lakes (bronze, silver, and gold layers).

Collaboration & Problem-Solving:

  • Experience working within agile development teams and applying DevOps best practices.
  • Strong problem-solving skills, with the ability to analyse and resolve complex data integration challenges.
  • Excellent communication and stakeholder engagement skills to translate business requirements into technical solutions.

Other

  • Security Clearance: Candidates must hold a valid Baseline security clearance or be willing and eligible to obtain one.
  • Location Requirement: This role is Canberra-based, and candidates must be available to work on-site at least 3 days per week (hybrid work arrangement).
  • Work Authorisation: Candidates must have Australian citizenship to be eligible for government security clearance.

Ready to take your data engineering career to the next level? Apply now and be part of a high-impact government transformation project!


Share this job

Interested in this job?
Save Job
Create As Alert

Similar Jobs

Read More
SCHEMA MARKUP ( This text will only show on the editor. )

JOB FINDER

OUR BLOG

By Toby Conran February 19, 2025
Recruitment is a tough game, especially when you're new to the industry. There's a lot to take in - KPIs to hit, industries to learn about, relationships to build, and cold calls to make. Your first few weeks and months as a recruiter can be overwhelming as you try to grow your expertise. In this blog, I cover some key tips that will have you kicking open doors in no time! Be Authentic Everyone has their own communication style and workflow. Find what works for you and stay true to it. You'll build stronger relationships with candidates and clients when you're professional while still letting your unique personality shine through. Be Confident in What You Offer Understand the value you bring as a recruiter and don’t be afraid to promote it. If you want your clients and candidates to stick around, you must believe that you’re the problem solver they need - and make sure they know it too. Establish a Process Early On Figure out what works best for you and build consistency into your daily workflow. Having a structured approach will help you stay organized and productive. Listen to Your Colleagues Your colleagues will be your best teachers in recruitment. Pay attention to how they handle different situations - how do they build rapport quickly? How do they deliver good (or bad) news? Listen to their questioning techniques and "borrow" the best ones to refine your own style. Learn from Your Candidates Use your conversations with candidates as a learning opportunity. Even the most complicated roles become clearer when you hear about them from different perspectives. Get on the Phone You don’t build strong relationships through emails alone. Prioritise in-person or phone-based communication to maximise your influence and establish real connections. Eat the Frog Tackle the least enjoyable task first thing in the morning. That way, you start the day on a high instead of procrastinating on that one difficult phone call. Be Consistent with Business Development Make Business Development (BD) a daily habit. Consistently building relationships and pipeline will ensure long-term success. Celebrate Successes Take time to acknowledge both your personal wins and team achievements. Whether it's securing that elusive client meeting or making your first placement, celebrating successes is key to avoiding burnout. "Recruitment is a marathon, not a sprint - stay authentic, stay consistent, and success will follow."

CLIENT RESOURCE PORTAL

Information, toolkits & advice to help

you on your pathway to success.

Access Client Resources
Share by: