Experience 15+Years
Job Description:
Looking for a data professional with expertise in data engineering, programming and architect to build systems that collect, manage, and convert raw data into usable information for business requirements. As a Data Engineer architect, you'll play a crucial role in ensuring data reliability, quality, and efficiency within the organization.
Responsibilities:
Analyze and organize raw data: Work with various data sources, document parsing, extracting relevant information and structuring it for further processing.
Build data systems and pipelines: Construct robust data pipelines that facilitate data flow from source to Target.
Evaluate business needs and objectives: Understand the company's requirements and align data systems accordingly.
Interpret trends and patterns: Use your analytical skills to identify data patterns.
Conduct complex data analysis and report on results: Dive deep into data to extract meaningful information.
Prepare data for prescriptive and predictive modeling: Ensure data is ready for machine learning and statistical analysis.
Build algorithms and prototypes: Develop and test data processing algorithms.
Combine raw information from different sources: Integrate data from various systems.
Explore ways to enhance data quality and reliability: Continuously improve data processes.
Identify opportunities for data acquisition: Stay informed about new data sources.
Develop analytical tools and programs: Create tools to facilitate data analysis.
Collaborate with data scientists and architects: Work closely with other data professionals to achieve common goals.
Implement data access controls, data encryption, and data masking techniques
Familiarity with data visualization tools and techniques for presenting data
Create and maintain dashboards and reports for stakeholders.
Mandatory Skills:
- 15+ years in Data Engineering / Data Architecture
Good knowledge of programming languages (e.g., Python, Java, Spark, etc).
Design, develop, and maintain data pipelines.
Exposure to process automation.
Experience working with REST APIs and services, messaging and event technologies.
Experience working with large and complex data sets.
Hands-on experience with SQL/No-SQL database (RDS, Redshift, DynamoDB, synapse, big query, mongo, etc.)
Batch/stream data processing experience
Monitor, troubleshoot, and optimize the performance of data infrastructure to ensure scalability, reliability, and cost efficiency.
Stay up to date with cloud services and best practices in data engineering to continuously improve our data ecosystem.
Good exposure on at least two public cloud platforms (Azure/AWS/Google Cloud Platform)
Experience with Graph database (e.g. Neptune, RDF4j, etc.)
Experience with Vector database (e.g. Pinecone, FAISS, etc.)
Hands-on experience with ETL/ELT tools (Informatica, Talend, dbt, etc.) - Expertise in cloud platforms: AWS / Azure / Google Cloud Platform - Experience with big data technologies: Spark, Hadoop, Kafka - Strong knowledge of data warehousing (Snowflake, Redshift, BigQuery) - Experience in building real-time and batch data pipelines - Strong understanding of data governance and security frameworks - Experience with CI/CD, DevOps practices for data pipelines - Proven leadership and stakeholder management skills
Good-to-Have Skills:
- knowledge or work experience in insurance, mortgage, banking domains.
- Proficiency in building stream processing systems using kinesis, Kafka, etc.
- Familiarity with Docker, Kubernetes, CI/CD and cloud services (AWS, Azure, Google Cloud Platform).
- Technical expertise with segmentation techniques.
- NLP knowledge
Reshu Yadav
Lead Recruiter
Phone-
Never miss an opportunity! Create an alert based on the job you applied for.
