Overview
Remote
$100,000 - $140,000
Full Time
No Travel Required
Skills
Amazon Web Services
Kubernetes
Python
Data Engineering
Scala
Cloud Computing
Big Data
Job Details
This position is 100% remote. No travel required.
- Design, develop and maintain manageable and scalable data pipelines and ETL processes to support data acquisition, integration and delivery for upstream applications, analytics, and other data-driven initiatives
- Design and implement advanced data storage solutions (databases, data warehouses, and data lakes) and efficient data transport layers (REST API s, message queues, message buses)
- Collaborate with executive leadership, product owners, technical architects, development teams, governance stakeholders, and other key personnel to gather, define, and document technical requirements for data pipelines that align with business objectives
- Create and maintain analytics dashboards providing actionable insights into customer acquisition, operational efficiency, and other critical business performance metrics
- Continuously optimize data architectures, data transformations, and data pipelines to enhance performance, reliability and scalability of the system.
- Apply software engineering best practices to ensure data quality, integrity, validity, availability, and compliance
- Proactively identify, troubleshoot, and resolve production data-related issues
- Create high-impact data tools for analytics and data science teams that help others move faster
- Participate in design, code development, code reviews, testing, data quality monitoring, deployment activities, and operations support
- Contribute to your overall team growth by staying current with and evaluating emerging data technologies and industry trends while sharing with colleagues
Qualifications
Required
- Bachelor s or Master s degree in computer science, Information Technology, Statistics, or a related field
- 5+ years of experience in data engineering with a modern tech stack
- Strong proficiency in Python, Java, or Scala
- Advanced SQL skills and extensive experience with relational databases (e.g., PostgreSQL, MS SQL Server) and experience with dbt is strongly preferred
- Hands-on experience with AWS and cloud data warehouses (e.g., Redshift, ClickHouse)
- Expertise in building batch and streaming data integration solutions (e.g., Kafka, Spark Streaming), data replication (e.g., AWS DMS, Airbyte, Debezium), and data transformation and enrichment (Python, SQL, Spark)
- Experience with data orchestration and workflow tools such as Airflow, Prefect, or AWS Step Functions
- Solid understanding of data modeling, data architecture, and data governance principles
- Exceptional problem-solving abilities, attention to detail, and strong communication and collaboration skills
- Prior experience designing, implementing, and supporting data pipeline architectures which include remote data ingestion, data orchestration, data governance, and data transformations.
Nice-to-Have
- Experience with containerization and orchestration tools (Docker, Kubernetes)
- Proficiency in Infrastructure as Code (IaC) technologies (e.g., Terraform, CloudFormation)
- Familiarity with CI/CD tooling and workflows
- Understanding of machine learning and data science workflows
- Extensive experience with big data technologies, focusing on data quality and observability
- Knowledge of NoSQL databases (e.g., MongoDB, Elasticsearch, Cassandra)
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.