Overview
Full Time
Part Time
Accepts corp to corp applications
Contract - W2
Contract - Independent
Skills
Programming Languages
Java
Spring Framework
API
SQL
Data Manipulation
Big Data
Apache Kafka
Streaming
Database
Routing
Scalability
Apache Velocity
Computer Cluster Management
Optimization
Microservices
Issue Resolution
Data Flow
Apache NiFi
Data Integration
Celery
Kubernetes
Workflow
Management
Performance Tuning
Orchestration
Scheduling
Job Details
ROLE: Java Tech lead ( Nifi, Airflow)
Description:
" Strong NiFi Expertise: Deep understanding of Apache NiFi architecture, processors, controllers, and data flow management capabilities.
Programming Languages: Proficiency in Java, Spring boot & API Microservices.
Database and SQL: Familiarity with databases, SQL, and data manipulation techniques.
Big Data Technologies: Kafka ( Event driven & streaming architecture)
Apachae Airflow - Workflow orchestration"
" Design and Implementation:
o Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes.
Ingestion and Transformation:
o Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption.
Routing and Enrichment:
o Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location).
Scalability and Performance:
o Ensuring data flows are scalable and performant to handle varying data volumes and velocity.
NiFi Cluster Management:
o Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting.
Microservices Integration:
o Developing and integrating microservices to support data conditioning, format validation, and transformation processes.
Troubleshooting and Issue Resolution:
o Identifying and resolving issues related to data flows, NiFi configurations, and data integration processes
Airflow Expertise:
o Possessing deep knowledge of Airflow's architecture, including schedulers, executors (Celery, Kubernetes), and plugin development.
Workflow Design and Development:
o Designing and developing complex, modular, and reusable DAGs (Directed Acyclic Graphs) to automate data pipelines.
Performance Optimization:
o Identifying and addressing performance bottlenecks in Airflow environments and implementing best practices for orchestration and scheduling
Description:
" Strong NiFi Expertise: Deep understanding of Apache NiFi architecture, processors, controllers, and data flow management capabilities.
Programming Languages: Proficiency in Java, Spring boot & API Microservices.
Database and SQL: Familiarity with databases, SQL, and data manipulation techniques.
Big Data Technologies: Kafka ( Event driven & streaming architecture)
Apachae Airflow - Workflow orchestration"
" Design and Implementation:
o Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes.
Ingestion and Transformation:
o Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption.
Routing and Enrichment:
o Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location).
Scalability and Performance:
o Ensuring data flows are scalable and performant to handle varying data volumes and velocity.
NiFi Cluster Management:
o Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting.
Microservices Integration:
o Developing and integrating microservices to support data conditioning, format validation, and transformation processes.
Troubleshooting and Issue Resolution:
o Identifying and resolving issues related to data flows, NiFi configurations, and data integration processes
Airflow Expertise:
o Possessing deep knowledge of Airflow's architecture, including schedulers, executors (Celery, Kubernetes), and plugin development.
Workflow Design and Development:
o Designing and developing complex, modular, and reusable DAGs (Directed Acyclic Graphs) to automate data pipelines.
Performance Optimization:
o Identifying and addressing performance bottlenecks in Airflow environments and implementing best practices for orchestration and scheduling
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.