Role Title: AWS Integration - DevOps/Cloud Engineers
Location: Hybrid - 2 to 3 days office - Knutsford
Duration: 9 Months
Rate: £ 400 per day - Umbrella Only
Role Overview:
Some onsite work will be required - Radbroke (London and Glasgow may also be options).
Deploy comprehensive cloud infrastructure for various products, including Astronomer Airflow and AccelData environments.
Role Duties:
- Facilitate cross-functional integration between vendor products and other systems, such as data lakes, storage, and compute services.
- Establish best practices for cloud security, scalability, and performance.
- Manage and configure vendor product deployments, ensuring the setup and maintenance of environments.
- Ensure high availability, scalability, and fault tolerance of Airflow clusters.
- Implement monitoring, alerting, and logging for Airflow and related components.
- Perform upgrades and patches for platform-related components.
- Oversee capacity planning, resource allocation, and optimization of Airflow workers.
- Maintain and configure integrations with source control systems (e.g., GitHub, GitLab) for version control.
- Collaborate with cloud providers (e.g., AWS) for pipeline integration and scaling requirements.
- Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimising data delivery, and automating manual processes.
- Develop infrastructure for optimal extraction, transformation, and loading of data from various sources using AWS and SQL technologies.
- Work with stakeholders, including design, product, and executive teams, to address platform-related technical issues.
- Build analytical tools to leverage the data pipeline, providing actionable insights into key business performance metrics, such as operational efficiency and customer acquisition.
