Data Engineer

  • Location

    Hove

  • Sector:

    Oil & Gas, Utilities

  • Salary:

    Starting from £56,000

  • Published:

    about 1 month ago

  • Expiry date:

    2020-11-17

  • Client:

    #

The company:

At EDF, we’re committed to helping our customers in any way we can. We are responsible for maximising the long-term value of our residential and business customers; we create an integrated approach to energy sourcing, pricing strategy, and deliver to the needs of our customers. We sell electricity and gas to over 5.5 million residential and business customers, making us the largest supplier of electricity by volume. We also run customer-facing activities including field sales and meter reading services. We’re committed to ‘getting it right first time’, making sure that service is continually improved & our approach to Customers is driven by our sustainability agenda, which means we behave like a socially and environmentally responsible company. We are the largest business unit within EDF with over 8,000 employees spread across 50 locations. Working in a competitive marketplace means we need high performing people who have the skills, experience, systems knowledge and leadership abilities to provide support and guidance in line with our ‘right-first-time’ policy.

#WeAreEDF and are leading the transition to a cleaner, low emission electric future, tackling climate change and helping Britain achieve net zero.   

The Person

The right candidate will work within an agile team of BigData Engineers building highly resilient, scalable and performant data engineering pipelines in a fully automated and governed AWS infrastructure. They will work alongside Data Scientists, BigData Application Developers, DevOps Engineers, and cross-functional IT teams. The right candidate will have extensive experience in statistical research, data engineering and the design of new algorithms and the application of statistical programming tools and techniques to large datasets.

The Data Engineer is expected to assume the following key responsibilities plus any other reasonable duties as required:

  • Build and manage a robust ETL/ELT pipelines for reading, cleaning, cataloguing, encrypting and delivery of disparate datasets.

  • Work with data governance and data quality teams to ensure compliance and quality.

  • Work with structured and unstructured data from variable source to ensure that data scientists working as part of product teams are provided with required high-quality data.

  • Ensure that data adhere to the enterprise standards relating to security, governance and documentation.

  • Ensure data are backed up and recovery plans are fully implemented.

  • Support testing of the data processing pipelines, ETL/ELT processes and algorithms.

  • Update operational and support documentation.

The Skills

  • Extensive hands-on experience in Python, Pandas, PySpark 

  • Expert in creating ETL/ELT pipelines, Data Cataloguing, Data Lineage, Data Warehousing

  • Excellent working knowledge of Big Data stack: HDFS, YARN, Hive, Apache Spark

  • Experience with major RDBMS: PostgreSQL, MySQL, MS SQL

  • Orchestration tools: Apache Airflow, Oozie, AWS Step Functions, etc.

  • Expert in statistical tools, techniques and practices employed in engineering BigData

  • DevOps mindset, excellent troubleshooting/debugging skills

  • Several years of experience working in leading Cloud Platforms such as AWS, Azure, Google, etc.

Highly Desirable

  • SQL Certification

  • Linux or Cloud (AWS, Azure, GCP etc.) Certification

  • Experience operating in a highly regulated industry with high-security standards

  • Understanding of Agile methodologies such as Scrum, Kanban etc.

  • Basic understanding of Data Science algorithms and methods

Competitive Salary and Benefits

We offer a competitive salary and benefits package, including a pension scheme, a wide range of flexible benefits to suit your lifestyle, and the opportunity to earn bonus.  

These roles are advertised on a full-time/part-time basis as standard with the opportunity to discuss flexible working arrangements.

Why EDF?

At EDF, everyone’s welcome. It’s a powerful statement of our intent to make sure we’re about the people who work with us – a rich and unique mix of backgrounds and experiences that’s vital to shaping a strong and fresh-thinking organisation like ours. Whoever you are and wherever you come from, we’ll embrace the difference you bring and give you opportunities to thrive and succeed.

Join us. Together, we’ll help Britain achieve net zero