Data Solution Architect with Datastage, Databricks , healthcare

HMG America LLC is the best Business Solutions focused Information Technology Company with IT consulting and services, software and web development, staff augmentation and other professional services. One of our direct clients is looking for Data Solution Architect with Datastage, Databricks , healthcare in Newtown square, PA. Below is the detailed job description.

Position: Data Solution Architect with Datastage, Databricks , healthcare

Location: Newtown square, PA (5 days onsite)

Fulltime role

Job Description:

As a Solution Architect, you will be responsible for:

• Strategic planning and hands-on engineering of Big Data and cloud environments that support our clients' advanced analytics and data science initiatives.
• Provide support in defining the scope and sizing of work
• Working closely with various enterprise architects Information security teams, Data management team, to ensure the architected solution meets all the needs of a customer, from a functionality perspective and IT solution engineering perspective.
• Translate business requirements into technology solutions
• Lead and own all aspects of our data solution including artifact creation such as diagrams, playbooks and other technical documents
• Mentor and guide Jr. team members to deliver the solutions on-time
• Create various architecture blueprints and work with the development team to deliver the vision.

Domain/Functional Skills:

Technical Skills:

• Overall 10+ years of experience of experience with Data Management, Big Data, Data Warehousing and Analytics.
• Must have Databricks, DataStage experience and healthcare domain exp.
• Experience with Azure services to create a medallion architecture for greenfield and brownfield implementations.
• Experience in exposing databricks unity catalog based datasets for REST API consumption.
• At least 3 to 4 years of experience in Architecting and migrating Oracle, DataStage based Enterprise warehouse to Azure using Databricks.
• 2-3 years of experience in designing and developing ETL solutions using Data Stage.
• Understanding and reverse engineering of DataStage parallel ,server and sequence ETL jobs to Databricks SparkSQL/PySpark/Spark Declarative pipelines.
• Analysis and converting custom stages, local containers, shared containers, routines and functions to Databricks SparkSQL/PySpark code.
• Analysis of the DataStage Configuration & Batch-job Shell scripts.
• Deep expertise in reverse engineering of Data Stage code and producing complexity reports.
• 3+ years of hands-on experience in creating an end-to-end lineage of all inbound and outbound file/data delivery including regulatory or operational reporting
• 3+ years of hands-on experience with ADF, Pyspark
• 1-2 years of experience with Tidal.
• One to two years of experience in Collibra DQ and DG is plus.
• Hands on working experience with scripting languages such as Python, Pyspark, Spark SQL or Shell Scripting, etc.
• Extensive hands-on experience in building and optimizing Databricks notebooks, tuning and optimizing ETL/ELT process developed ADF.
• Experience with implementation of data security, encryption, PII/PSI legislation, identity and access management across sources and environments.
• Experience with data process Orchestration, end-to-end design and build process of Near-Real Time and Batch Data Pipelines
• 3-5 years of experience in healthcare Payer domain experience is a nice to have experience.
• Ability to provide technical leadership and oversight on multiple projects/tracks.
• Strong client-facing communication and facilitation skills