Overview
Remote
$65
Contract - W2
Contract - 12 month(s)
Job Details
Hello,
Responsibilities:
Position: Java NIFI Technical SME
Location: Raleigh NC
Contract
Strong NiFi expertise, including architecture, processors, controllers and data flow management.
Experience with data integration, ETL processes and data modelling concepts
Proficiency in programming languages such as Java, Python or other relevant scripting languages
Familiarity with databases, SQL and data manipulation techniques
Knowledge of big data technologies like Kafka
Responsibilities:
Create and implement data flow solutions using NiFi, defining data sources, destinations and transformation processes.
Manage data ingestion from various sources (files, databases, APIs) and transform data into desired formats for downstream consumption.
Configure NiFi to route data to appropriate destinations and enrich data with additional information such as timestamps and location.
Ensure data flows are scalable and performant to handle varying data volumes and velocity.
Maintain and improve NiFi clusters, including configuration, optimization and troubleshooting.
Develop and integrate microservices to support data conditioning, format validation and transformation processes.
Possess deep knowledge of Airflows architecture, including schedulers, executors (Celery, Kubernetes) and plug-in development.
Design and develop complex, modular and reusable DAGs to automate data pipelines.
Identify and address performance bottlenecks in Airflow environments and implement best practices for orchestration and scheduling.
Integrate Airflow with cloud-native services such as Azure Data Factory, Azure Databricks, Azure Storage and others.
Develop and maintain CI/CD pipelines for Airflow DAG deployment, testing and version control, often using tools like Azure DevOps.
Identify and resolve issues related to data flows, NiFi configurations and data integration processes.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.