Big Data Lead with Azure || Philadelphia, PA || Fulltime or W2 // Hybrid

Overall 14+ years of Azure & Data Engineering experience Design and develop cloud-native applications and services using modern architecture patterns. Collaborate with product managers, developers, and other stakeholders to define application requirements and specifications.

Strong hands-on experience in Database design, ETL with primary skills on Azure Databricks - Build data Pipelines using ADB - leveraging Python and PySpark, ADF: Azure Data Factory, Azure Data Lake Gen 2, Synapse Analytics.

Good experience and knowledge in scripting languages such as Python, PySpark, Spark SQL or Shell Scripting, etc. In-depth understanding of various storage services offered by Azure and TIDAL Scheduler.

Experience with implementation of data security, encryption, PII/PSI legislation, identity and access management across sources and environments.

Experience with data process Orchestration, end-to-end design and build process of Near-Real Time and Batch Data Pipelines.

Minimum 3 years of real project experience. Evaluate and recommend cloud technologies and tools to enhance application performance and reliability.

Lead the migration of existing applications to cloud environments, ensuring minimal disruption and optimal performance. Implement best practices for cloud security, compliance, and governance. Monitor and optimize application performance, scalability, and cost-efficiency in cloud environments.

Provide technical leadership and mentorship to development teams, fostering a culture of innovation and continuous improvement.

Stay current with emerging cloud technologies and trends and assess their potential impact on our architecture and applications.

Provide Technical Assistance, Problem Resolution and Troubleshooting Support.