senior data engineer

Job Title: Senior Data Engineer
Location: Remote

Job Description:

We are looking for an experienced Senior Data Engineer to join our healthcare client's Data and Analytics organization. The ideal candidate will have strong expertise in cloud-based data engineering, data pipelines, ETL processes, and data modeling, preferably within Google Cloud Platform or Azure environments.

The candidate will be responsible for designing, developing, and maintaining scalable data engineering solutions that support analytics, reporting, and data science initiatives.

Responsibilities:

• Design, develop, and maintain scalable cloud-based data pipelines and ETL processes.

• Build and optimize data models, data marts, and database systems to support analytics and data science.

• Develop large-scale data structures and pipelines for data collection, transformation, integration, and standardization.

• Work with cross-functional teams, including data scientists, analysts, business SMEs, and software engineers.

• Implement batch and real-time data processing solutions.

• Develop and optimize queries for efficient data processing and retrieval.

• Integrate data from multiple sources while ensuring data quality, consistency, and accessibility.

• Collaborate with data science teams to integrate machine learning models into automated workflows.

• Implement CI/CD-enabled data pipelines and follow cloud architecture best practices.

• Evaluate new tools and technologies to improve data engineering solutions.

• Design and develop APIs and microservices for real-time data ingestion and integration.

Required Qualifications:

• 7+ years of progressively complex experience in data engineering.

• Strong programming skills in Python, Java, or similar languages.

• Experience with PySpark, Apache Spark, Pig, MySQL, NoSQL, or similar technologies.

• Hands-on experience with cloud platforms such as Google Cloud Platform or Azure.

• Experience developing ETL pipelines, data models, and data integration solutions.

• Strong understanding of database architecture and query optimization.

Preferred Qualifications:

• Experience with Google Cloud Platform services, including BigQuery, Cloud Storage, Dataproc, Cloud Composer, and Cloud Shell.

• Experience with Azure cloud data engineering services.

• Knowledge of Apache Airflow, Kafka, Spanner, HDFS, Hive, and Spark.

• Experience with CI/CD pipelines and automation.

• Knowledge of Bash scripting, Unix utilities, and Linux commands.

• Experience with real-time data ingestion and API/microservice development.

• Exposure to machine learning model implementation and Tableau.

• Strong analytical, problem-solving, communication, and collaboration skills.

• Experience building high-volume data pipelines and solving complex data engineering challenges.

Education:

• Bachelor's degree in Computer Science, Engineering, Machine Learning, or a related STEM discipline, or equivalent work experience.

• Master's degree or PhD preferred.

Additional Preference: