Data Engineer
The data engineer will be integral in designing and implementing data solutions for STA's portfolio of digital tests. You will lead on the development and implementation of data storage solutions and digital data flows. You will also provide learning and development to existing team members to upskill the team.
Working within the Standards and Testing Agency, an executive agency of a overnment department, you will work with colleagues in the Assessment Development, Data and Digital teams as well as external contractors to apply your specialist knowledge of data engineering to the ongoing development of new digital systems.
The role:
- Scope the requirements of current and future systems, as well as working with colleagues within the department to understand and align with the department's data strategy,
- Develop data validation processes throughout data flow, working with colleagues to ensure their data needs are met, and deliver testing and assurance of data systems,
- Implement data storage solutions (e.g. data warehousing) for existing digital platforms and any future developments, ensuring solutions are in line with departmental data strategy
- Upskill existing team members in order to grow capability and resilience in this area.
Key Skills / Experience
- Have experience of leading, up skilling and motivating a team in a multi-disciplinary environment, delivering products within specific time- scales and working alongside multiple senior stakeholders
- Be an accomplished ETL developer who has proven experience of delivering complex big data solutions and a track record of working with structured and unstructured data.
- Have extensive working knowledge of Data Warehouse techniques for dimensional data modelling, relational database design, ETL techniques and data analysis
- Have knowledge of data architecture concepts / technologies e.g. data warehouse, data lakes, big data
- Have expert level knowledge of data reconciliation, ETL error handling and error logging mechanisms
- Have advanced coding skills in SQL and knowledge of JSON or XML
- Have experience using R to extract, transform and validate data.
