The Data Engineer Senior designs, builds, and maintains scalable data pipelines and core data infrastructure to ingest, process, store, and transform structured and unstructured data in a highly regulated client environment. This role develops and operates automated, end-to-end pipelines, optimizes data models and schemas, and ensures the reliability, scalability, and security of the enterprise data platform. The engineer also implements robust data quality, validation, and lineage mechanisms and collaborates with cross-functional stakeholders to align data architecture with business and mission needs while supporting continuous improvement of data engineering practices.
Key Responsibilities- Design and implement distributed data pipelines using frameworks such as Apache Spark, Kafka, or Flink to support high-volume, near-real-time and batch data processing across the enterprise.
- Develop and optimize data models, schemas, and storage patterns across relational and non-relational platforms (e.g., PostgreSQL, Snowflake, S3, data lakes) to enable analytics, reporting, and downstream system integration.
- Implement and operate ETL/ELT workflows using orchestration tools such as Apache Airflow or cloud-native services, ensuring end-to-end automation, observability, and recoverability.
- Establish and maintain data quality validation rules, monitoring, and anomaly detection processes to protect data integrity across production pipelines.
- Build and maintain data lineage, cataloging, and metadata management solutions to support governance, auditability, and regulatory compliance in a highly regulated environment.
- Apply security best practices, including encryption, role-based access controls, and compliance alignment, when handling sensitive and mission-critical data.
- Troubleshoot and resolve complex pipeline failures, performance bottlenecks, and data inconsistencies, driving root-cause analysis and long-term remediation.
- Collaborate with data scientists, analysts, and application teams to translate analytical and integration requirements into scalable data engineering solutions and reusable data assets.
Required Qualifications- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent relevant experience in data engineering or software engineering.
- Typically 5-8 years of experience in data engineering or closely related roles, including hands-on work with distributed data processing, data modeling, and ETL/ELT implementation.
- Demonstrated experience with modern data engineering frameworks and tools such as Apache Spark, Kafka or similar streaming platforms, and workflow orchestrators like Apache Airflow.
- Strong proficiency with relational and non-relational data stores (e.g., PostgreSQL, cloud data warehouse platforms, object storage) and with designing schemas for analytics and integration use cases.
- Experience implementing data quality, validation, and lineage capabilities in production environments, including monitoring and alerting for data pipeline health.
- Ability to obtain and maintain a SECRET-level background investigation, with U.S. citizenship required to support federal client environments.
- Solid understanding of data security principles, including encryption, access control, and secure handling of sensitive information in compliance-driven environments.
Preferred Qualifications- Experience designing and operating data platforms on major cloud providers such as AWS, Azure, or Google Cloud, including use of native data and analytics services.
- Professional certification in cloud data engineering or big data technologies (e.g., AWS Data Analytics, Google Professional Data Engineer, Azure Data Engineer Associate).
- Prior experience supporting mission-critical or highly regulated government systems where data governance, auditability, and compliance requirements are central.
- Experience contributing to the evolution of enterprise data architectures, standards, and best practices, including documentation of data pipelines and participation in data governance forums.
Compensation RangesCompensation ranges for ASM Research positions vary depending on multiple factors; including but not limited to, location, skill set, level of education, certifications, client requirements, contract-specific affordability, government clearance and investigation level, and years of experience. The compensation displayed for this role is a general guideline based on these factors and is unique to each role. Monetary compensation is one component of ASM's overall compensation and benefits package for employees.
EEO RequirementsIt is the policy of ASM that an individual's race, color, religion, sex, disability, age, sexual orientation or national origin are not and will not be considered in any personnel or management decisions. We affirm our commitment to these fundamental policies.
All recruiting, hiring, training, and promoting for all job classifications is done without regard to race, color, religion, sex, disability, or age. All decisions on employment are made to abide by the principle of equal employment.
Physical Requirements
The physical requirements described in "Knowledge, Skills and Abilities" above are representative of those which must be met by an employee to successfully perform the primary functions of this job. (For example, "light office duties' or "lifting up to 50 pounds" or "some travel" required.) Reasonable accommodations may be made to enable individuals with qualifying disabilities, who are otherwise qualified, to perform the primary functions.
DisclaimerThe preceding job description has been designed to indicate the general nature and level of work performed by employees within this classification. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications required of employees assigned to this job.
Never miss an opportunity! Create an alert based on the job you applied for.