Principal Platform Data Engineer (Databricks Platform)

Overview

Remote
On Site
Hybrid
BASED ON EXPERIENCE
Full Time

Skills

PySpark
Specification Gathering
Extract
Transform
Load
Storage
Scalability
Data Quality
Real-time
Analytics
Batch Processing
Performance Tuning
Data Engineering
Big Data
Computer Science
Software Engineering
Streaming
Programming Languages
Python
Java
Scala
Data Processing
Apache Spark
Apache Flink
Version Control
Git
Continuous Integration
Continuous Delivery
Workflow
Data Warehouse
Data Modeling
Relational Databases
NoSQL
Database
Amazon Web Services
Cloud Computing
Databricks
Regulatory Compliance
Problem Solving
Conflict Resolution
Analytical Skill
Communication
Collaboration
FSA
Network
FOCUS
Professional Services
Genetics
Law

Job Details

Principal Platform Data Engineer (Databricks Platform)

Title: Principal Platform Data Engineer (Databricks Platform)
Location: Remote
Duration: Permanent/FTE
Compensation: $180,000.00-$200,000.00
Work Requirements: , Holders or Authorized to Work in the U.S.

WHAT THIS ROLE WILL DO
This is a hands-on Data role with strong Python/PySpark coding skills

  • Collaborate with stakeholders to understand and translate business requirements into technical specifications and designs.
  • Design, develop, and maintain scalable data pipelines, ETL processes, and data warehouses to support data ingestion, transformation, and storage.
  • Evaluate and select appropriate technologies, tools, and frameworks for data processing, storage, and analytics, considering security, scalability, performance, and cost factors.
  • Define and enforce data engineering best practices, standards, and guidelines to ensure data quality, reliability, and consistency.
  • Architect and implement solutions for real-time data processing, streaming analytics, and batch processing.
  • Conduct performance optimization, tuning, and troubleshooting of data infrastructure components to ensure optimal performance and resource utilization.
  • Stay abreast of emerging trends, technologies, and best practices in data engineering, cloud computing, and big data analytics.
  • Consistent exercise of independent judgment and discretion in matters of significance.
  • Participate in on-call rotations/Pagerduty for data platform support.
  • Other duties and responsibilities as assigned.
WHAT THIS PERSON WILL BRING: TECHNICAL SKILLS/COMPETENCIES
  • Bachelor's or Master's degree in Computer Science, Engineering, or related field.
  • 12+ years of software engineering experience, with a focus on designing and implementing data platforms.
  • Proven track record of successfully architecting and delivering large-scale data solutions, e.g. platforms that process millions of streaming events per second or Petabytes of data daily.
  • Strong proficiency in programming languages such as Python, Java, or Scala.
  • Hands-on experience with data processing frameworks like Apache Spark or Apache Flink.
  • Hands on experience using version control systems such as Git and CI/CD workflows and practice
  • Extensive knowledge of data warehousing, data modeling, relational databases, and NoSQL databases.
  • Hands-on experience with AWS and cloud-based data services and solutions (e.g., Databricks).
  • Hands-on experience with data compliance (eg. CCPA, GDPR, etc) and governance
  • Excellent problem-solving skills, analytical thinking, and the ability to thrive in a fast-paced, dynamic environment.
  • Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.

BENEFITS & PERKS
Our motto is 'Taking Care of Our Own' through 6 pillars of benefits:
  • HEALTH: Medical, Vision and Dental benefits for you and your family, including Flexible Spending Accounts (FSA) and Health Savings Accounts (HSAs)
  • YOURSELF: Generous paid time off policy including paid holidays, sick time and paid days off for your birthday, Free concert tickets
  • WEALTH: 401(k) program with company match, Stock Program
  • FAMILY: New parent programs & support including caregiver leave and childcare cash, infertility support
  • CAREER: Tuition reimbursement, student loan repayment internal growth and development programs & trainings
  • OTHERS: Volunteer time off, crowdfunding network

About INSPYR Solutions
Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com.
INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About INSPYR Solutions