Overview
Skills
Job Details
Role: Sr. Data Engineer(8+) Location: Remote
Any visa is fine
Need health care client experience
Key Responsibilities:
Design, build, and maintain robust ETL pipelines using Python and Py Spark.
Work with large-scale structured and unstructured data from various sources including databases, APIs, and flat files.
Optimize data workflows for performance and scalability in distributed computing environments.
Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
Ensure data quality, integrity, and consistency across all data pipelines.
Monitor and troubleshoot ETL jobs and data flows.
Implement best practices for data engineering, including version control, testing, and documentation.
Required Skills & Qualifications:
Bachelor's or master's degree in computer science, Engineering, or related field.
3+ years of experience in data engineering or ETL development.
Strong programming skills in Python.
Hands-on experience with PySpark and distributed data processing using Apache Spark.
Proficiency in SQL and working with relational databases (e.g., PostgreSQL, MySQL, SQL Server).
Experience with cloud platforms (e.g., AWS, Azure, Google Cloud Platform) and data services (e.g., S3, Redshift, Big Query).
Familiarity with workflow orchestration tools like Apache Airflow, Luigi, or Prefect.
Knowledge of data warehousing concepts and dimensional modeling.
Excellent problem-solving and communication skills.