Duties:
Work with business and technical leadership to understand requirements.
Design to the requirements and document the designs.
Ability to write product-grade performant code for data extraction, transformations and loading using Spark, Py-Spark.
Do data modeling as needed for the requirements.
Write performant queries using Teradata SQL, Hive SQL and Spark SQL against Teradata and DataBricks Unity Catalog.
Implementing dev-ops pipelines to deploy code artifacts on to the designated platform/servers like AWS or DataBricks.
Implement DataBricks job orchestration using DataBricks.
Troubleshooting the issues, providing effective solutions and jobs monitoring in the production environment.
Participate in sprint planning sessions, refinement/story-grooming sessions, daily scrums, demos and retrospectives
Skills:
Strong development experience in Spark, Py-Spark, Shell scripting, Teradata, and DataBricks.
Experience of Ab Initio is required.
Strong experience in writing complex and effective SQLs (using Teradata SQL, Hive SQL and Spark SQL) and Stored Procedures.
Excellent work experience on DataBricks as data warehouse/Data Lake implementations.
Experience in Agile and working knowledge on DevOps tools (Git, Jenkins, Artifactory) Unix/Linux Shell scripting (KSH) and basic administration of Unix servers CA7 Enterprise Scheduler AWS (S3, EC2, SNS, SQS, Lambda, ECS, Glue, IAM, and CloudWatch) Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration).
Experience in Jira and Confluence Exercises considerable creativity, foresight, and judgment in conceiving, planning, and delivering initiatives.
Health care domain knowledge is a plus
Education: .
Bachelor’s degree in a field such as Computer Science/Engineering
Required Skills:
Spark
Py-Spark
Shell Scripting
Teradata
AWS
SQL
Databricks
Health Care Domain Knowledge
Additional Skills:
Jenkins
Artifactory
Unix/Linux Shell Scripting
Stored Procedures
Ab Initio
Sql
Teradata Sql
Hive Sql
Spark Sql
Git
Unix Server Administration
Ca7 Enterprise Scheduler
AWS
S3
EC2
SNS
SQS
Lambda
ECS
GLUE
IAM
Cloudwatch
Databricks Delta Lake
Databricks Notebooks
Databricks Pipelines
Databricks Cluster Management
Data Lake Implementation
AGILE
Data Warehouse Implementation
Devops Tools
Azure/Aws Integration
JIRA
Confluence
Never miss an opportunity! Create an alert based on the job you applied for.