Expert in SQL and Python for developing and debugging scalable data pipelines.
Deep hands-on experience with Azure and Databricks, including Delta Live Tables and Unity Catalog
JOB DESCRIPTION
The Senior Data Engineer, leads the design and implementation of robust data solutions across multiple domains, driving technical excellence and scalability. This role mentor others, shape best practices, and influence data architecture. This role is expected to proactively identify opportunities to improve systems, drive reliability, and collaborate with product and business stakeholder to again data strategy and company goals.
Profile Description:
Design, build and scale robust, high-performing batch and real-time data pipelines. Drive architectural decisions for transformation logic, storage formats, and schema design. Lead complex data ingestion efforts and mentor peers on performance optimization and scalability. Lead the design and optimization of complex data models and storage architecture, balancing performance, scalability, and usability. Partner with stakeholders to translate business requirements into robust data structures. Contribute significantly to delivery planning and execution, mentor junior engineers on agile approaches, and ensure timely completion of tasks by managing dependencies and escalating delivery challenges. Design and standardize advanced data validation frameworks and testing strategies across platforms. Lead root cause analysis and data quality issues and mentor others on quality best practices. Partner with stakeholders to define SLAs and quality metrics. Leads efforts to automate, monitor, and scale deployment of production-grade data pipelines. Design resilient workflows with retry logic, failure handling, and resource optimization. Proactively address performance and reliability issues and contribute to runbooks and on-call documentation. Lead the creation and maintenance of detailed technical documentation for complex pipelines, data models, and system integrations. establish and enforce documentation and development standards across projects. Mentor junior engineers on clear, consistent coding and documentation habits. Act as a key technical partner to product, analytics and data science teams. Lead design, discussions, communicate complex data trade-offs with clarity, and proactively surface risks and blockers. Support collaborative planning and mentor junior team members in effective communication and partnership.
Knowledge & Experience: 5-9 years in data engineering, data modeling, and pipeline development Expert in SQL and Python for developing and debugging scalable data pipelines. Deep hands-on experience with Azure and Databricks, including Delta Live Tables and Unity Catalog Skilled with data integration/orchestration tools (SnapLogic, Azure Data Factory, Jenkins) Strong use of infrastructure-as-code tools like Terraform to manage deployment pipelines. Design and optimize API integration in pipelines. Familiar with data quality observability tools such as Soda or similar. Proficient in version control and CI/CD workflows using GitHub. Advanced understanding of dimensional modeling and data warehousing concepts. Comfortable leading efforts in agile environments and strong ownership and collaboration.
Regards,
Vinay Ram
(Direct)
Suwanee, GA - 30024
An MBE & eVerify Company
Connect with me for exciting career opportunities:
Open Jobs (For Recruiters):
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Senior Data Engineer Remote prefer candidates form Charleston, SC Long Term W2 The Senior Data Engineer, leads the design and implementation of robust data solutions across multiple domains, driving technical excellence and scalability. This role mentor others, shape best practices, and influence data architecture. This role is expected to proactively identify opportunities to improve systems, drive reliability, and collaborate with product and business stakeholder to again data strategy and com
Job Description Lead data design, prototype, and development of data pipeline architecture pipelines. Lead implementation of internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability. Lead cause analysis on external and internal processes and data to identify opportunities for improvement and answer questions. Excellent analytic skills associated with working on unstructured datasets. Understand the architecture, be
Credit Acceptance is proud to be an award-winning company recognized both locally and nationally across multiple workplace categories. Our world-class culture is shaped by dedicated team members who are driven to succeed as professionals individually and together as a team. Backed by a strong product, exceptional people, and a stable financial foundation, we've grown into a leading provider of used and new car financing across the country. Our Engineering and Analytics Team Members utilize the
At Allstate, great things happen when our people work together to protect families and their belongings from life's uncertainties. And for more than 90 years, our innovative drive has kept us a step ahead of our customers' evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection. Job Description Job Description Allstate's Data & Analytics Technology or
Never miss an opportunity! Create an alert based on the job you applied for.
