Data Engineering Lead - (Scripting - Docker - Jira - Data pipelines) - Remote - 6 months
Our client, a market leading multi-national technology company are looking for a Data Engineering Lead, to join them on a customer based project based in Remote. The assignment is an initial 3 month contract and can offer a competitive daily rate via Umbrella
The role is for an enthusiastic, passionate Data Engineering Leads, who are passionate about complex data sets, can derive quick insights with precision and have ability to inter-relate the data patterns and business needs.
Job Description:
- Architect a complete Datalake project/workstream.
- Define and be responsible for the architecture of the end to end solution.
- For each component in an end to end solution:
- Select appropriate Blueprints/Design Patterns for each element of the solution.
- Design and document detailed designs.
- Assign detailed designs to appropriate Data Engineer for development and test.
- Facilitate a Show & Tell session with relevant stakeholders.
- Track handover to Operations.
- Where a requirement cannot be met by an existing Design Pattern:
- Provide a new Design Pattern for that requirement (and deliver as above).
- Ensure that Design Pattern is fully integrated into the Design Pattern Repository for future re-use.
- Be the primary Orvel contact point for the stakeholder community of that project/workstream.
- An effective communicator (both written and orally).
- Experience of setting, communicating, and tracking targets.
- Experienced in facilitating difficult discussions (both within the team or with senior stakeholders).
Key Responsibilities:
- Design scalable data pipelines.
- Design new API integrations.
- Architect and own data models.
- Drive a coherent technology roadmap from sometimes ambiguous requirements balancing a focus on long-term ambitious vision with short and medium-term customer requirements.
Bring technical experience and insight to the table and help the team navigate difficult trade-offs and pursue the right opportunities.
- Design, develop, optimize, and maintain data architecture and pipelines that adhere to ETL principles and business goals.
Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
- Produce test specifications and review/sign-off completed tests.
- Work closely with key stake holders to help formulate strategy and roadmaps.
- Work closely and collaboratively with the Orvel Scrum Master/Delivery Manager.
Produce and own the data platform architecture.
- Be the first point of contact and "go to person" for all members of the project/workstream team.
- Provide day to day support and coaching to the project team.
- Set and communicate targets.
Technical Skills:
- NoSQL (Elasticsearch)
- Data pipelines (Filebeat, MetricBeat, Logstash, Kafka)
- Data visualisation (Kibana)
- Jira, Git
- Docker,
- Scripting Languages (Python, Ruby)
- Kubernetes (Added advantage)
Data Engineering Lead - (Scripting - Docker - Jira - Data pipelines) - Remote - 6 months
