Senior Data Engineer (Python/Scala)

{"description": " The Work : ICF is looking for an enthusiastic Data Engineer to join our team and help with Data Management and Data Analysis. If you are Data Engineer interested in applying your expertise in Data Engineering in a consulting environment, then this may be the role for you.

Job Location: This position requires that the job be performed in the United States. If you accept this position, you should note that ICF does monitor employee work locations and blocks access from foreign locations/foreign IP addresses, and also prohibits personal VPN connections.

• You may be asked to travel once a quarter to an office.
• Our core work hours are 10am - 4pm Eastern Time with the option to start earlier or work later depending on your time zone. However, please note our client is on the east coast and may sometimes start a meeting earlier than 10:00 which may require your participation.

What You Will Do:

• Create Dashboards using AWS QuickSight with both visuals (charts, graphs, etc...) as well as tables for end users to slice and dice data to gain insights to various business processes.
• Design and maintain scalable Spark-based data ingestion pipelines with adaptive change management to accommodate evolving business needs and technical requirements.
• Lead centralized orchestration for both batch and event-driven workflows, ensuring seamless and efficient data movement throughout the platform.
• Develop reusable templates and self-service solutions to enable efficient updates and enhancements to data models, empowering teams to manage changes independently.
• Optimize distributed compute resources to enhance performance, reliability, and cost-effectiveness of data processing environments.
• Define and enforce data contracts, manage schema versioning, and automate metadata processes to uphold reliable data standards and strong governance.
• Collaborate in a federated model to operationalize essential compliance requirements, including handling personally identifiable information (PII), data retention, and maintaining consistent naming conventions across datasets.
• Enforce robust data quality checks-including schema validation, handling of nulls, uniqueness, volume, freshness, and distribution metrics-as well as referential integrity across all datasets.
• Embed orchestration of data quality checks at various checkpoints within the pipeline to ensure ongoing compliance and reliability.
• Log, audit, and measure all quality results to provide transparency, accountability, and continuous improvement in data quality management.
  Leadership & Execution
• Work with architects as a technical leader, contributing to the establishment of engineering standards, best practices, and guiding critical design decisions.
• Partner with business and domain owners to understand domain data structure and translate requirements into reliable and scalable data products.
• Lead incident triage, conduct root cause analysis, and drive continuous improvements in platform reliability and data quality.
• Define and track key performance indicators (KPIs) for data quality, freshness, stability, adoption, and cost.Demo work in both small and large virtual settings with clients and end users to obtain feedback on enhancing dashboards to meet business requirements.
• Work within a SAFe scaled agile framework, collaborating with other team members to ensure solutions meet client needs with the highest quality.

What You Will Bring With You:

• Bachelor's Degree
• 1+ years of experience working with tools like JIRA, GitHub, and Confluence.
• 2+ years of experience with working on cloud platforms in AWS.
• 2+ years of experience relational database and data warehousing concepts
• 1+ years of experience with python and Scala, Spark technologies
• 1+ years experience with data orchestration tools like NiFi, Airflow, Step Functions, etc.
• 1+ years of experience with serverless or cloud-native analytics platforms
• Candidate must be able to obtain and maintain a Federal Public Trust
• Candidate must reside in the U.S., be authorized to work in the U.S., and all work must be performed in the U.S.
• Candidate must have lived in the U.S. for three (3) full years out of the last five (5) years

What We Would Like You To Bring With You:

• Familiarity with data profiling, data catalogs, lineage tools, or observability platforms.
• Prior experience or knowledge in contributing to or leading federated data governance
• 5 years' excellent problem-solving skills and end-to-end quantitative thinking.
• Ability to self-organize, prioritize and conduct work on multiple projects under tight deadlines in a fast-paced environment.
• Prior experience in consulting or healthcare is an advantage but not essential.

Professional Skills:

• Good leadership and team-working skills.
• Highly effective analytical, problem-solving, and decision-making capabilities.
• Excellent communication and interpersonal skills to interface effectively at all levels of the business.
• Organized, detailed oriented and able to prioritize and multi-task.
• Travel for a conference or to another ICF location for collaboration may be required once a year. This position requires that the job be performed in the United States. If you accept this position, note that ICF monitors employee work locations, blocks access from foreign locations/foreign IP addresses, and prohibits personal VPN connections.

#DMX-HES

#Li-cc1

#Indeed

Working at ICF
ICF is a global advisory and technology services provider, but we're not your typical consultants. We combine unmatched expertise with cutting-edge technology to help clients solve their most complex challenges, navigate change, and shape the future.

We can only solve the world's toughest challenges by building a workplace that allows everyone to thrive. We are an equal opportunity employer. Together, our employees are empowered to share their expertise and collaborate with others to achieve personal and professional goals. For more information, please read our EEO policy.

We will consider for employment qualified applicants with arrest and conviction records.

Reasonable Accommodations are available, including, but not limited to, for disabled veterans, individuals with disabilities, and individuals with sincerely held religious beliefs, in all phases of the application and employment process. To request an accommodation, please email and we will be happy to assist. All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.

Read more about workplace discrimination rights or our benefit offerings which are included in the Transparency in (Benefits) Coverage Act.

Candidate AI Usage Policy

At ICF, we are committed to ensuring a fair interview process for all candidates based on their own skills and knowledge. As part of this commitment, the use of artificial intelligence (AI) tools to generate or assist with responses during interviews (whether in-person or virtual) is not permitted. This policy is in place to maintain the integrity and authenticity of the interview process.

However, we understand that some candidates may require accommodation that involves the use of AI. If such an accommodation is needed, candidates are instructed to contact us in advance at We are dedicated to providing the necessary support to ensure that all candidates have an equal opportunity to succeed.

Pay Range - There are multiple factors that are considered in determining final pay for a position, including, but not limited to, relevant work experience, skills, certifications and competencies that align to the specified role, geographic location, education and certifications as well as contract provisions regarding labor categories that are specific to the position.

The pay range for this position based on full-time employment is:
$98,614.00 - $167,644.00

Nationwide Remote Office (US99)", "salary_raw": "USD 98,614.00 - 167,644.00 per year"}