Overview
Skills
Job Details
Title : Lead Data Engineer WIth Nifi + Python
Location: Maryland Heights, MO
Experience: 10 Years
Contact: C2C/w2
Detailed JD:-
Skills:
Primary Skills Required
o Should have extensive experience in developing and managing Apache NiFi data flows
o Excellent hands-on experience with Python
o Expert Level knowledge in SQL
Secondary Skills Required:
o Data Engineering: Experience with data ingestion, ETL, and data warehousing.
o Databases: Knowledge of relational databases
o Cloud Platforms: Experience with AWS
o Data Modeling: Ability to design and implement data models for analytical and reporting purposes.
o Security: Understanding of data security principles and best practices.
o Communication and Collaboration: Excellent communication and collaboration skills.
Responsibilities:
NiFi Development:
o Designing, developing, and managing data flows between systems using the Apache NiFi platform
Data Flow Design and Implementation:
o Design ,document and implement data pipelines for various business needs, including data ingestion, transformation, and loading.
Data Pipeline Optimization:
o Optimize data flows for performance, scalability, and efficiency, addressing bottlenecks and improving processing times.
Troubleshooting and Debugging:
o Identify and resolve issues within data pipelines, including data quality issues, performance problems, and security vulnerabilities.
Data Quality and Security:
o Ensure data quality and security throughout the data pipeline, implementing appropriate data validation, cleansing, and security measures.
Leadership and Communication:
o Excellent communication, collaboration, and leadership skills.