TechOps-CloudOps-Airflow-Engineer-GDSN02

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. 

Service line - Position Title: TechOps-CloudOps - Senior Apache Airflow Platform Engineer

At EY, we’re all in to shape your future with confidence.  

We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.​  

Join EY and help to build a better working world.  

About Global Delivery Services

Global Delivery Services refers to EY's worldwide network of service delivery centers. The GDS team plays an important role in EY’s strategy by ensuring effective support to EY’s growth agenda.

Our journey started in 2002 with approximately 200 people. Today we stand at 80,000+ professionals in ten locations around the world. We operate in Argentina, China, Hungary, India, Philippines, Poland, Sri Lanka, Mexico, Spain and the United Kingdom.

Client service is focused on providing Consulting, Assurance, Tax, Strategy & Transactions, and Knowledge support to our clients around the world. The teams enable account teams worldwide to provide seamless, high-quality, value-added support, helping deliver exceptional client service.

Enablement Services provides cost-effective, high-skilled, and innovative services to support EY’s global and local enablement teams. Markets, BMC, AWS, Finance and Accounting, Risk Management, Procurement, People Shared Services, IT Service Delivery and IT Global Infrastructure services, are among the services offered by Enablement Services.

Our innovation specialists serve the GDS Client Service and Enablement Services teams, along with Service Lines, Core Business Services and Sectors. The team brings the desired environment, technologies and skilled teams together for facilitation, rapid prototyping and innovative thinking. The competencies offered include analytics, digital, user experience, mobile technology, infrastructure, Microsoft technologies and open innovation.

The Opportunity

We are seeking a highly skilled Senior Apache Airflow Platform Engineer to provide L2 operational support, platform administration, automation, and continuous improvement for enterprise workflow orchestration environments. The ideal candidate will possess strong expertise in Airflow DAG development, platform operations, DevOps automation, cloud-native infrastructure, and CI/CD implementation.

Your Key Responsibilities

This role will be responsible for ensuring the reliability, scalability, security, and performance of the Airflow platform while partnering closely with Data Engineering, Cloud Infrastructure, DevOps, and Application teams.

Airflow Platform Operations

• Provide L2/L3 support for Apache Airflow production environments.
• Monitor, troubleshoot, and resolve issues related to DAG execution, task failures, scheduling delays, and workflow dependencies.
• Support Airflow platform administration, configuration management, upgrades, and patching activities.
• Manage Airflow components including Schedulers, Webservers, Workers, Executors, Metadata Database, Variables, Connections, and Plugins.
• Perform root cause analysis for recurring workflow and platform incidents.
• Develop and maintain operational runbooks, SOPs, troubleshooting guides, and knowledge articles.
• Ensure platform availability, reliability, and performance through proactive monitoring and capacity planning.

Airflow Development & Orchestration

• Design, develop, and maintain Airflow DAGs for complex data and application workflows.
• Optimize DAG performance, scheduling efficiency, and resource utilization.
• Implement best practices for workflow orchestration, dependency management, and fault tolerance.
• Support onboarding of new workflows and integrations into the orchestration platform.
• Review DAG code and enforce development standards across engineering teams.

DevOps & CI/CD Engineering

• Design, implement, and maintain CI/CD pipelines using GitHub Actions.
• Automate deployment, testing, validation, and release management processes for Airflow environments.
• Implement code quality controls, automated testing, security scanning, and deployment governance.
• Support GitOps and Infrastructure-as-Code practices across platform deployments.
• Collaborate with development teams to streamline release and deployment processes.

Cloud & Infrastructure Automation

• Support and administer Airflow deployments on AWS EKS.
• Manage Kubernetes-based workloads and ensure application scalability and resilience.
• Develop and maintain Helm charts for deployment and configuration management.
• Build and manage Infrastructure-as-Code solutions using Terraform.
• Support containerized environments leveraging Docker and Kubernetes best practices.
• Implement automation for environment provisioning, monitoring, recovery, and operational maintenance.

Database Administration & Support

• Manage and support PostgreSQL databases used by Airflow and associated platforms.
• Monitor database health, performance, storage, and availability.
• Troubleshot metadata database issues affecting Airflow operations.
• Support backup validation, recovery procedures, and database maintenance activities.
• Optimize database performance through query tuning and configuration improvements.

Skills and attributes for success

• 3-5 years of experience supporting, administering, and engineering Apache Airflow platforms in enterprise environments.
• Good experience in Airflow DAG development, workflow orchestration, scheduling, dependency management, and production platform support.
• Deep understanding of Airflow architecture, including Schedulers, Webservers, Workers, Executors, Metadata Database, Plugins, Variables, Connections, and deployment frameworks.
• Experience troubleshooting DAG failures, task execution issues, scheduler bottlenecks, performance degradation, dependency conflicts, and platform stability issues.
• Hands-on experience designing, developing, and maintaining CI/CD pipelines using GitHub Actions, including automated testing, code validation, deployment automation, release management, and governance.
• Experience implementing DevOps and GitOps practices, software delivery lifecycle management, and Infrastructure-as-Code methodologies.
• Strong hands-on experience with AWS cloud services and cloud-native platform operations.
• Expertise in deploying and supporting Airflow on AWS EKS and Kubernetes-based environments.
• Experience creating, managing, and maintaining Helm Charts for application deployment and configuration management.
• Strong experience with Docker containerization and container lifecycle management.
• Proven expertise in Terraform for infrastructure provisioning, automation, and environment management.
• Strong administration and troubleshooting experience with PostgreSQL, including performance monitoring, query optimization, backup validation, recovery procedures, and metadata database management.
• Experience integrating Airflow with enterprise applications, APIs, databases, cloud services, data platforms, and messaging systems.
• Experience with enterprise monitoring and observability tools such as Grafana, Prometheus, CloudWatch, Datadog, Splunk, or equivalent platforms.
• Strong understanding of platform reliability engineering, operational monitoring, alerting frameworks, capacity planning, and performance optimization.
• Practical experience with Incident Management, Problem Management, Change Management, and Service Request Management processes.
• Experience performing root cause analysis (RCA), implementing corrective actions, and driving continuous service improvement initiatives.
• Experience creating and maintaining operational documentation, runbooks, SOPs, troubleshooting guides, and knowledge articles.
• Knowledge of enterprise security, access management, governance, audit, and compliance requirements within cloud-native environments.
• Experience with Amazon Managed Workflows for Apache Airflow (MWAA).
• Knowledge of AWS services including IAM, S3, Lambda, CloudWatch, Secrets Manager, and ECR.
• Experience with OpenShift and enterprise Kubernetes platforms.
• Experience with ServiceNow, Jira, Confluence, GitHub Enterprise, and Agile delivery methodologies.
• Knowledge of enterprise security, audit, governance, and compliance requirements.
• Responsible for decision – making, optimizing processes, resource management, and overseeing team management as needed for task execution.
• Accountable for allocating personnel, supervising team members, assigning tasks, ensuring that the team has the necessary tools and support to succeed in their roles, and optimizing and evaluating their performance to meet organizational goals.

Support Model

• Support coverage will be 18 X 5, with on-call support for critical incidents on a 24X7 basis.
• Main service hours will be 6 AM IST to 12 AM IST, Monday to Friday, excluding applicable national holidays in the support locations.
• On-call service hours will cover 12 AM IST to 6 AM IST, Monday to Friday, and 24 hours on weekends for critical and high-priority incidents.
• Out-of-hours L1.5 incident triage may be handled through a shared service desk, while ticket resolution and SOP execution will remain within the agreed support team scope.
• SLA tracking will depend on ticket ownership and will pause where resolution is pending with client, vendor, infrastructure, database, network, or third-party teams.
• Support volumes, ticket trends, incident recurrence, and workload distribution will be reviewed periodically to identify capacity or process improvement needs.
• Manual monitoring and repetitive operational activities should be reviewed for automation opportunities during steady-state support.
• Operational Level Agreements will be defined with interfacing teams where platform dependencies exist.

What we offer you

At EY, we’ll develop you with future-focused skills and equip you with world-class experiences. We’ll empower you in a flexible environment, and fuel you and your extraordinary talents in a diverse and inclusive culture of globally connected teams. Learn more. 

Are you ready to shape your future with confidence? Apply today.  

To help create an equitable and inclusive experience during the recruitment process, please inform us as soon as possible about any disability-related adjustments or accommodations you may need.

Our Commitment: As a commitment, we persistently endeavour to embody our values, fulfil our purpose, and champion inclusiveness. Our dedication is to cultivate EY into an environment where diverse perspectives are celebrated, creating a supportive atmosphere for individuals to authentically be themselves and contribute their utmost.

Professional Development: From entry-level employees to senior leaders, we believe in continuous learning. We offer opportunities to build new skills, take on leadership roles, and connect and grow through mentorship.

People and Culture: In our dynamic workplace, diversity, equity, and inclusiveness are ingrained in our culture. We're united by a commitment to create an environment where every individual's differences are valued, practices are equitable, fostering a sense of belonging. Our shared values include integrity, respect, teaming, inclusiveness, energy, enthusiasm, courage to lead, and building relationships based on doing the right thing.

Benefits: Embark on a transformative career journey with us and indulge in a suite of premium benefits, encompassing exclusive health and wellness packages, enticing rewards, and cutting-edge learning opportunities that empower you to continually grow and excel in your professional and personal development.

How to Apply: If you are passionate to join us and are aligned with our commitment to building a better working world, we invite you to apply by completing our user-friendly form with personal and professional information and by providing your consent to data privacy.

Successful candidates advance to a competency-based interview. If mutual interest persists, a job offer awaits.

“We are an equal opportunity employer and are committed to Diversity, Equity & Inclusion”.

EY | Building a better working world 

 
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.  

 
Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.  

 
Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.