Databricks Architect

Job Title: Databricks Architect

Location: Remote

Job Type : Fulltime / Contract

Job Description :

• Evaluates the introduction of New Skills and implements solutions and enterprise data integration strategies
• Provide analysis and recommendations to overall data strategies and roadmaps
• Partners with Solution Architects to perform source system analysis, identification of key data issues, data profiling and development of normalized and star-snowflake physical schemas
• Helps lead engineering teams and guides members in best practices for data movement, data quality, master data management (MDM), data masking, data sub-setting and/or data virtualization development (collectively referred to as data integration and data management activities)
• Drives the technical design, installation & configuration, development, optimization, error handling and support of data management initiatives for customer engagements
• Installs and configures software in on-premises and cloud environments
• Ensures the compliance of project team to development, naming, and coding standards
• Creates and implements scheduling strategies for data management processes
• Administers and supports data management environments and repositories
• Works with business partners to review prototypes and develop iterative revisions
• Oversees and performs troubleshooting and provides resolutions for reporting issues
• Works closely with and assists other consultants, provides leadership, and collaborates with clients to understand and interpret data through stakeholder interviews and by defining, analyzing, and validating data
• Develops appropriate tool development guidelines and documentation
• Evaluates and tests new tools and technologies.
• Develops and executes test cases/plans to ensure high-quality releases, based on test driven development using industry best practices such as Gherkin
• Defines data pipeline orchestration and subsequently automates code deployments or mentors junior team members to automate the deployments using tools such as Apache Airflow, Control-M, or Autosys
• Performs data management development tasks in areas of data movement, data quality, MDM, business intelligence, etc. - DQ, data architecture, etc.
• Supports testing, root cause analysis, bug fixing and deployment activities
• Design and implement various development, testing, automation, CI/CD tools, and cloud infrastructure
• Set up tools and required infrastructure across various services such Snowflake, AWS, SnapLogic and Alation.
• Monitor product roadmaps for different providers and components and recommend enhancements according to product updates.

30% Communication/Organization/Documentation/Leadership

• Collaborate and ensures success with offshore delivery team
• Drafts detailed documentation (i.e., Design Specs, Test Plans, etc.) that have a professional appearance
• Assists in producing detailed and comprehensive documentation for testing and deployment activities.
• Deals effectively with all team members and builds strong working relationships/rapport with them
• Demonstrates ability to organize tasks and time necessary to complete assigned tasks/deliverables
• Coordinates with Project Managers and Leads to help drive project planning
• Perform feasibility analysis, scoping exercises, and works with the project manager to prioritizes deliverables
• Leads the data engineering aspect of projects for the enterprise, including hands on delivery
• Leads the software development team during the design, build, test, support phases
• Directs and implements data integration processes, including data capture, testing and validation methods
• Prioritize and organize multiple tasks, providing direction, tools, technical and process support to developers
• Provides appropriate training, mentoring and support for Data integration tools to the developers
• Provides proactive technical oversight and advice to integration architect and development team
• Assists developers, analysts, and designers in conceptualizing and validating solutions that meet business requirements
• Aligns data architecture plans with strategic business objectives
• Utilizes agile-based approach to implementing data services and solutions
• Provide analysis and recommendations to overall data strategies and roadmaps
• Coordinates and oversee the assignments, delivery, and quality of data integration project
• Develops appropriate tool development guidelines and documentation
• Ensures the quality, completeness, security, privacy, and integrity of data throughout the data lifecycle
• Assists with infrastructure and capacity planning
• Assist with the development and enforcement of integration standards

Education & Experience

• 5+ years of experience in a Data Management/Engineer role
• 1+ years of consulting experience strongly preferred
• Practical experience in at least 3 of the following areas:
• Practical development experience with an object-oriented programming language
• Python and Java preferred
• Experience with version control frameworks such as Git and Subversion preferred
• Highly proficient in SQL
• Proficient at writing complex SQL statements and interpreting results
• Proven experience in operational analysis, data analysis, and problem resolution activities
• Proficient with ETL design and development
• Drafts detailed documentation (i.e., Design Specs, Test Plans, etc.) that have a professional appearance
• Excels in an agile environment where critical thinking and strong problem-solving skills are required for success
• Experience with developing and supporting data management environments utilizing tools such as Tableau, Power BI, TIBCO, MSBI, Informatica, and Databricks
• Experience with or ability to learn modern data platforms involving AWS, Azure, Google Cloud Platform, Snowflake, Denodo, etc. Ability to learn MDM platforms,
• Experience with Informatica MDM, Orchestra EBX, or similar tools a plus
• Bachelor's degree or Equivalent
• Excellent organizational and analytic abilities
• Outstanding communication and people skills
• Effective decision-making skills