EY - GDS Consulting - AI And DATA - Data Engineer - Manager

Job Title: Data Engineer -Manager

EY is a global leader in assurance, tax, transaction and advisory services. Technology is at the heart of what we do and deliver at EY. Technology solutions are integrated in the client services we deliver and are key to our innovation as an organization. 

Fueled by strategic investment in technology and innovation, Client Technology seeks to drive growth opportunities and solve complex business problems for our clients through building a robust platform for business and powerful product engine that are vital to innovation at scale. As part of Client Technology, you’ll work with technologists and business experts, blending EY’s deep industry knowledge and innovative ideas with our platforms, capabilities, and technical expertise. As a catalyst for change and growth, you’ll be at the forefront of integrating emerging technologies from AI to Data Analytics into every corner of what we do at EY. That means more growth for you, exciting learning opportunities, career choices, and the chance to make a real impact.

EY Technology:

Technology has always been at the heart of what we do and deliver at EY. We need technology to keep an organization the size of ours working efficiently. We have 250,000 people in more than 140 countries, all of whom rely on secure technology to be able to do their job every single day.  Everything from the laptops we use, to the ability to work remotely on our mobile devices and connecting our people and our clients, to enabling hundreds of internal tools and external solutions delivered to our clients.  Technology solutions are integrated in the client services we deliver and is key to us being more innovative as an organization.

EY Technology supports our technology needs through three business units:

• Client Technology (CT) focuses on developing new technology services for our clients. It enables EY to identify new technology-based opportunities faster and pursue those opportunities more rapidly.
• Enterprise Workplace Technology (EWT) – EWT supports our Core Business Services functions and will deliver fit-for-purpose technology infrastructure at the cheapest possible cost for quality services. EWT will also support our internal technology needs by focusing on better user experience.
• Information Security (Info Sec) - Info Sec prevents, detects, responds and mitigates cyber-risk, protects EY and client data, and our information management systems. 

The opportunity

As the Data Engineering Lead, you will be responsible for the architectural design, development, and maintenance of our enterprise data platforms on AWS. You will lead a high-performing team of engineers, foster a culture of technical excellence while utilize Scrum methodologies to ensure timely delivery of data products.

Your key responsibilities

• Technical Leadership: Lead the design and implementation of scalable, automated data pipelines using PySpark and AWS services.
• Team Management: Manage, mentor, and upskill a team of data engineers. Conduct code reviews and ensure adherence to best practices in software engineering.
• Agile Delivery: Act as the Scrum Lead/Manager for the data team. Organize and facilitate daily stand-ups, sprint planning, and backlog grooming within Jira.
• Stakeholder Collaboration: Translate complex technical concepts into actionable insights for non-technical stakeholders across teams.

Nice to have

Architecture & Strategy: Define the long-term data strategy, ensuring the architecture supports big data processing, real-time analytics, and machine learning requirements, if required.

Data Quality & Governance: Implement robust monitoring, logging, and data quality frameworks to ensure high reliability and "single source of truth" integrity.

Skills and attributes for success

Mandatory Technical Skills

• AWS Ecosystem - Expert knowledge of S3, Glue, EMR, Redshift, Lambda, Iceberg, Kinesis, and Athena. Experience with IAM and VPC security is a plus.
• Big Data Processing - Advanced PySpark skills, including performance tuning, partitioning strategies, and handling skewness in large datasets.
• Advanced SQL - Mastery of complex analytical queries, window functions, and database performance optimization.
• CI/CD & DevOps - Hands-on with automating deployments via Jenkins or GitHub Actions. Exposure to Terraform or CloudFormation is a plus
• Data Modelling - Expertise in Kimball/Inmon methodologies and designing Data Lakes/Lakehouse’s.

Leadership & Soft Skills

• Scrum Mastery: Proven experience managing data projects in Jira, tracking sprint velocity, and removing blockers for the team.
• Communication: Exceptional verbal and written communication skills; ability to influence senior leadership and navigate technical ambiguity.
• Strategic Thinking: Ability to balance "perfect" technical solutions with "pragmatic" business deadlines.

To qualify for the role, you must have

• 8+ years of experience in Data Engineering, Data Architecture, or a related field.
• 8+ years in a formal leadership or management role.
• Bachelor’s or master’s degree in computer science, Engineering, or a related quantitative field.
• Relevant certifications (e.g., AWS Certified Data Engineer - Associate/Professional) are highly preferred.
• Deep expertise in Databricks (PySpark, Delta Lake, notebooks)
• Strong programming skills in Python and PySpark
• Experience with GitHub, GitHub Actions, and Terraform for AWS deployments
• Proficiency in Medallion Architecture and Kimball modeling
• CI/CD implementation for data workflows
• Hands-on experience implementing batch and streaming pipelines in Databricks using DAGs
• Familiarity with data mesh principles and advanced data architecture patterns
• Develop and maintain efficient ETL (Extract, Transform, Load) pipelines that ingest, process and transform large datasets from diverse sources.
• Build and manage data storage solutions using relational and non-relational databases, ensuring high availability, performance and scalability.
• Work with APIs, third-party data sources and internal databases to create unified datasets that provide actionable insights for business stakeholders.
• Partner with business intelligence analysts, data scientists and software engineers to design and implement data-driven solutions that meet business requirements.
• Design optimized data schemas and structures that support analytical workloads and improve query performance.
• Leverage orchestration tools like Apache Airflow or Prefect to automate ETL workflows and improve operational efficiency.
• Provide technical guidance, review code and promote best practices within the data engineering team.

Primary skillset:

• Python
• Proficient in SQL - Stored procedure, Optimization techniques for fine tuning the data hydration pipelines
• Proficiency in DB - DB2/Oracle/SQL server
• Data modeling
• AWS-S3,Glue, AWS EMR and SNS
• AWS Redshift, Lambda
• PySpark
• Shell scripting
• Apache Airflow
• Deployment of pipelines using CI/CD pipelines.

Secondary skillset

• AWS Step Functions
• AWS Event bridging scheduling
• Docker
• AWS ECS/EKS
• NoSQL - AWS DynamoDB.
• Realtime data streaming - Kafka/AWS Kinesis.

Ideally, you’ll also have

• Strong verbal and written communication, facilitation, relationship-building, presentation and negotiation skills.
• Be highly flexible, adaptable, and creative.
• Comfortable interacting with senior executives (within the firm and at the client)

What we look for

Strong teamwork, work ethic, product mindset, client centricity and a relentless commitment to EY values.

What working at EY offers

We offer a competitive remuneration package where you’ll be rewarded for your individual and team performance. Our comprehensive Total Rewards package includes support for flexible working and career development, and with FlexEY you can select benefits that suit your needs, covering holidays, health and well-being, insurance, savings and a wide range of discounts, offers and promotions. Plus, we offer:

• Support, coaching and feedback from some of the most engaging colleagues around
• Opportunities to develop new skills and progress your career
• The freedom and flexibility to handle your role in a way that’s right for you