EY - GDS Consulting - AI And DATA - AWS DE - Gig

Job Description –AWS Data Engineer

Objectives and Purpose

The Senior Data Engineer ingests, builds, and supports large-scale data architectures that serve multiple downstream systems and business users. This individual supports the Data Engineer Leads and partners with Visualization on data quality and troubleshooting needs.

The Senior Data Engineer will:

• Clean, aggregate, and organize data from disparate sources and transfer it to data warehouses.
• Support development testing and maintenance of data pipelines and platforms, to enable data quality to be utilized within business dashboards and tools.
• Create, maintain, and support the data platform and infrastructure that enables the analytics front-end; this includes the testing, maintenance, construction, and development of architectures such as high-volume, large-scale data processing and databases with proper verification and validation processes.

Your key responsibilities

• Must have 3 to 5 years of experience with Python and Data Engineering.
• Hands on experience with AWS ( Athena, Redshift)
• Hands on experience with AWS Glue, Pyspark, Python
• Hands on experience with AWS Lambda
• Expert-level experience in creating/analyzing complex SQL and creating data pipelines, data modeling skills, and frameworks.
• Expert-level experience in creating/analyzing complex SQL and creating data pipelines, data modeling skills, and frameworks.
• Strong technical knowledge and extensive application support experience in systems and applications within clinical development
• Problem-solving, analytical, quality mindset, and strong business acumen
• Ability to translate complex business questions and requirements and develop effective solutions.
• Understanding of clinical data standards
• Excellent analytical skills
• Detail-oriented, demonstrating good interpersonal and leadership skills, a team player.
• Self-motivated, work effectively under pressure.
• Effective verbal and written communication skills

Required Skills & Qualifications

• Experience: 3-5 years of progressive experience in Data Engineering, Data Warehousing, or a related role.
• Programming: Expertise in Python for data manipulation, scripting, and pipeline development.
• Data Transformation: Strong proficiency with modern SQL
• Big Data Ecosystem: Hands-on experience with PySpark and distributed processing frameworks, specifically on AWS Glue, Lambda, Pyspark
• Cloud Data Warehousing: Practical experience working with and optimizing large datasets in cloud data warehouses such as Amazon Redshift
• Cloud Storage: Experience with Amazon S3 for building and managing data lakes.
• DevOps/CI/CD: Familiarity with CI/CD tools like Jenkins and source control systems, specifically Bitbucket (or similar like Git/GitHub/GitLab).
• Methodology: Proven experience working effectively within an Agile - SCRUM development framework.
• Communication: Exceptional written and verbal communication skills with the ability to influence and collaborate across teams.

Desired skillsets

• Knowledge of data visualization tools (e.g., Tableau, Power BI).
• Familiarity with containerization technologies (e.g., Docker, Kubernetes).
• Bachelor's or master’s degree in computer science, Engineering, or a related field.