Big Data Engineer

Title:-Senior Big Data Engineer

Location:-Washington, DC(Onsite)

Long Term Contract

Position Overview

We are seeking an experienced Senior Big Data Developer to design, develop, and optimize enterprise data platforms, data pipelines, and analytics solutions. This individual will play a key role in expanding and enhancing data architecture capabilities, enabling efficient ingestion, organization, processing, and visualization of large-scale data assets. The ideal candidate combines strong data engineering expertise with hands-on experience in database design, data modeling, cloud technologies, and scalable data processing frameworks.

Key Responsibilities

• Design, develop, and maintain enterprise data architectures, databases, data lakes, and scalable data platforms.
• Build, optimize, and automate ETL/ELT workflows and data integration pipelines.
• Develop and maintain scalable data processing frameworks capable of handling high-volume structured and unstructured data.
• Collaborate with business stakeholders, analysts, researchers, and technical teams to understand data requirements and deliver effective solutions.
• Improve existing data systems and recommend enhancements to increase performance, reliability, and scalability.
• Design and implement optimal data delivery architectures to support analytics, reporting, forecasting, and operational needs.
• Perform root cause analysis on data issues and business processes to identify opportunities for improvement.
• Support migration of data pipelines and workflows between on-premises and cloud environments.
• Implement data governance, quality, security, and operational best practices.
• Contribute to software development lifecycle activities, including development, testing, deployment, and maintenance of data applications and services.
• Design and communicate enterprise information architecture at conceptual, logical, and physical levels.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field.
• Minimum of 7 years of experience in data engineering, data architecture, database administration, or related disciplines.
• Advanced proficiency in SQL and experience with relational database platforms such as PostgreSQL, Microsoft SQL Server, and MySQL.
• Advanced programming experience with Python, R, and other scripting languages used for data engineering and analytics.
• Experience working with large-scale data systems, distributed computing environments, and scalable data processing architectures.
• Experience designing, developing, and automating ETL/ELT workflows and data integration solutions.
• Experience building, optimizing, and maintaining scalable databases and data pipelines.
• Strong software engineering background, including development, testing, and deployment of data-driven applications.
• Experience working in Linux environments and utilizing source control platforms such as GitHub and GitLab.
• Excellent analytical, troubleshooting, problem-solving, verbal communication, and written communication skills.
• Strong customer-service orientation and ability to work independently in a collaborative environment.

Preferred Qualifications

• Advanced degree in a technical discipline.
• Experience with workflow orchestration and automation tools such as Apache Airflow, Prefect, Dagster, or AWS Step Functions.
• Experience with cloud platforms including AWS, Microsoft Azure, and Snowflake.
• Experience implementing data warehouses utilizing Change Data Capture (CDC) methodologies.
• Experience implementing and maintaining CI/CD pipelines and DataOps practices.
• Experience with NoSQL and graph database technologies.
• Experience developing, training, deploying, and maintaining machine learning models.
• Knowledge of time-series data management, analytics, and forecasting techniques.
• Experience integrating structured and unstructured data sources.
• Working knowledge of additional programming languages such as Java, Scala, JavaScript, or Perl.
• Experience supporting analytics, research, financial, scientific, or data-intensive environments.

Core Competencies

• Data Architecture & Modeling
• Big Data Engineering
• Cloud Data Platforms
• ETL/ELT Development
• Database Design & Administration
• Data Integration & Automation
• DataOps & CI/CD
• Machine Learning Enablement
• Enterprise Information Architecture
• Performance Optimization & Troubleshooting
• Cross-Functional Collaboration
• Strategic Problem Solving

Strategic Problem Solving