Role Title: Hadoop Engineer / ODP Platform
Location: Birmingham / Sheffield - Hybrid working with 3 days onsite per week
End Date: 28/11/2025
Role Overview:
We are seeking a highly skilled Hadoop Engineer to support and enhance our Operational Data Platform (ODP) deployed in an on-premises environment.
The ideal candidate will have extensive experience in the Hadoop ecosystem, strong programming skills, and a solid understanding of infrastructure-level data analytics. This role focuses on building and maintaining scalable, secure, and high-performance data pipelines within enterprise-grade on-prem systems.
Key Responsibilities:
- Design, develop, and maintain data pipelines using Hadoop technologies in an on-premises infrastructure.
- Build and optimise workflows using Apache Airflow and Spark Streaming for real-time data processing.
- Develop robust data engineering solutions using Python for automation and transformation.
- Collaborate with infrastructure and analytics teams to support operational data use cases.
- Monitor and troubleshoot data jobs, ensuring reliability and performance across the platform.
- Ensure compliance with enterprise security and data governance standards.
Required Skills & Experience:
- Minimum 5 years of experience in Hadoop and data engineering.
- Strong hands-on experience with Python, Apache Airflow, and Spark Streaming.
- Deep understanding of Hadoop components (HDFS, Hive, HBase, YARN) in on-prem environments.
- Exposure to data analytics, preferably involving infrastructure or operational data.
- Experience working with Linux systems, shell scripting, and enterprise-grade deployment tools.
- Familiarity with monitoring and logging tools relevant to on-prem setups.
Preferred Qualifications:
- Experience with enterprise ODP platforms or similar large-scale data systems.
- Knowledge of configuration management tools (e.g., Ansible, Puppet) and CI/CD in on-prem environments.
- Understanding of network and storage architecture in data centers.
- Familiarity with data security, compliance, and audit requirements in regulated industries.
