EY - GDS Consulting - AI And DATA - Databricks Solution Architect - Associate Director

EY- GDS - Data and Analytics – Associate Director – Databricks Solution Architect

As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance.

The opportunity

We’re looking for a candidate with 18+ years with strong architect expertise into Databricks Solution Architect and delivery lead

Role Purpose

The Databricks Solution Architect is responsible for designing, implementing, and governing scalable data, analytics, and AI/ML solutions using the Databricks Lakehouse platform. This role ensures alignment between business requirements and technical architecture while enabling high-performance, secure, cost-optimized data and machine learning platforms across cloud environments.

Key Responsibilities

• Design end-to-end Databricks Lakehouse architectures including ingestion, processing, analytics, and AI/ML workloads.
• Architect batch and streaming data pipelines using Apache Spark, Delta Lake, and Databricks Workflows.
• Design and govern ML lifecycle workflows using MLflow, Feature Store, Model Registry, and Model Serving.
• Enable production-grade AI/ML pipelines including training, experiment tracking, deployment, and monitoring.
• Define platform standards, architectural patterns, and best practices for data and AI workloads.
• Optimize cluster configurations, job performance, ML training workloads, and cost management strategies.
• Implement data governance, security, lineage, and access controls using Unity Catalog and cloud IAM.
• Collaborate with data scientists, ML engineers, data engineers, and business stakeholders to translate AI use cases into scalable solutions.
• Support CI/CD pipelines for data and ML workflows including automated testing and model deployment.
• Provide technical leadership, architectural reviews, and solution guidance across delivery teams.

Required Skills & Experience

• Strong hands-on experience with Databricks platform components including Delta Lake, Unity Catalog, SQL Warehouses, MLflow, Feature Store, and Model Serving.
• Expertise in Apache Spark, PySpark, Spark SQL, and distributed processing for analytics and ML workloads.
• Experience building and operationalizing machine learning pipelines on Databricks.
• Proficiency in Python and SQL with strong performance tuning and data modeling skills.
• Experience with cloud platforms (AWS, Azure, or GCP) including storage, compute, networking, and security services.
• Knowledge of MLOps practices such as experiment tracking, model versioning, CI/CD for ML, and model monitoring.
• Understanding of AI/ML frameworks (such as scikit-learn, TensorFlow, PyTorch) and their integration with Databricks.
• Experience designing enterprise-scale data and AI architectures.
• Strong communication, stakeholder management, and technical leadership skills.

Nice to Have

• Experience with autonomous systems, reinforcement learning, or AI copilots
• Industry exposure across BFSI, Manufacturing, Healthcare, Retail, or Telecom

Ideally, you’ll also have

• Client management skills

What we look for

• People with technical experience and enthusiasm to learn new things in this fast-moving environment

What working at EY offers

At EY, we’re dedicated to helping our clients, from start-ups to Fortune 500 companies — and the work we do with them is as varied as they are.

You get to work with inspiring and meaningful projects. Our focus is education and coaching alongside practical experience to ensure your personal development. We value our employees, and you will be able to control your own development with an individual progression plan. You will quickly grow into a responsible role with challenging and stimulating assignments. Moreover, you will be part of an interdisciplinary environment that emphasizes high quality and knowledge exchange. Plus, we offer:

• Support, coaching and feedback from some of the most engaging colleagues around
• Opportunities to develop new skills and progress your career
• The freedom and flexibility to handle your role in a way that’s right for you