Our client is a British multinational mining company with headquarters in London. They are the world's largest producer of platinum, as well as being a major producer of diamonds and other precious metals.
An opportunity has arisen for an experienced Data Engineer to join their team on a 6 month contract basis. Please note that this role will be initially remote working with some travel to London once office working resumes.
Data Engineer Lead Responsibilities:
- Develop, test, operate and support the end-to-end batch, near real time data engineering solution
- Be able to develop quality Python/PySpark and adopt/implement software development best practices to ensure high quality standards are met
- Performs thorough code reviews of fellow developers and ensure quality code
- Use DevOps knowledge, cloud and data engineering expertise to operate, monitor and troubleshoot the pipelines / code bases
- Effectively test and adopt testing frameworks to embed and codify tests
- Take technical ownership for the implementation of the DE solution at an allocated mining site
- Clearly communicate and explain concepts to senior stakeholders and key teams during sprint reviews and other programme meetings
- Have a clear understanding of agile best practices and proactively collaborate and support the team if needed during the whole sprint cycle
- Document technical implementation details and artefacts created as part of the process
Python, PySpark, version control (Git), performance engineering, Azure cloud analytics services (ADF, Databricks, SQLDB, Event Hub
Cloud - Databricks , Microsoft AzureDevOps, DevOps, Data Modelling, Test Automation