Position: Senior Incorta Data Engineer

Experience Required

6-8 Years

Job Summary

We are seeking an experienced Senior Incorta Data Engineer to design, develop, and optimize enterprise reporting and analytics solutions using the Incorta platform. The ideal candidate should possess strong expertise in Incorta schemas, dashboards, Spark SQL, PySpark, SQL, ETL, data warehousing, and ERP data integration (SAP/Oracle).

The candidate will be responsible for building scalable data pipelines, integrating data from multiple enterprise sources, developing business dashboards, optimizing query performance, and ensuring high-quality reporting solutions in a large-scale data environment.

Key Responsibilities

Incorta Development

• Design, develop, and maintain Incorta schemas, Business Views, and Insights Dashboards.
• Build scalable and high-performing analytics solutions using the Incorta platform.
• Develop interactive dashboards and reports for business users.
• Optimize Incorta query performance and data load processes.
• Manage Incorta data models and schema design.

Data Engineering & Pipeline Development

• Design and build scalable ETL/data pipelines using PySpark.
• Develop data ingestion frameworks for structured and semi-structured data.
• Process large-scale datasets efficiently in distributed environments.
• Integrate data from databases, APIs, flat files, ERP systems, and cloud sources.
• Ensure data quality, consistency, and reliability throughout the data lifecycle.

Data Warehousing & Data Modeling

• Design enterprise data warehouse solutions.
• Develop logical and physical data models.
• Create optimized schemas for analytics and reporting.
• Apply dimensional modeling techniques and ETL best practices.
• Support enterprise reporting and business intelligence initiatives.

ERP & Enterprise Data Integration

• Integrate data from:
  • SAP
  • Oracle
  • SQL Databases
  • REST APIs
  • Flat Files
  • Enterprise Applications
• Build reliable integration pipelines for business-critical data.
• Ensure seamless synchronization between ERP systems and analytics platforms.

Performance Optimization

• Optimize SQL and Spark SQL queries.
• Improve data load performance within Incorta.
• Monitor system performance and identify bottlenecks.
• Enhance scalability and efficiency of data processing workflows.

Testing & Data Validation

• Perform data validation and reconciliation.
• Troubleshoot ETL and reporting issues.
• Validate business data accuracy and completeness.
• Support testing of dashboards and reporting solutions.

Collaboration

• Collaborate with business stakeholders to gather reporting and analytics requirements.
• Work closely with Data Architects, BI Developers, and Business Analysts.
• Participate in Agile ceremonies and project planning.
• Provide technical support and mentoring to junior developers.

Documentation

• Maintain technical documentation for:
  • Data models
  • ETL pipelines
  • Incorta schemas
  • Dashboard designs
  • Reporting solutions
• Create operational runbooks and deployment documentation.

Required Skills

Incorta

• Incorta Platform
• Schema Development
• Business Views
• Insights Dashboards
• Dashboard Development
• Spark SQL

Data Engineering

• PySpark
• Data Pipeline Development
• Data Transformation
• Data Processing
• Data Ingestion
• Data Integration

Data Warehousing

• Data Warehousing Concepts
• ETL Processes
• Data Modeling
• Schema Design
• Star Schema
• Snowflake Schema

Big Data

• SQL
• Spark SQL
• Large Dataset Processing
• Distributed Data Processing

ERP Integration

• SAP Integration
• Oracle Integration
• Database Integration
• REST APIs
• Flat File Processing

Performance & Quality

• Query Performance Optimization
• Data Load Optimization
• Data Validation
• Testing
• Troubleshooting
• Performance Monitoring
• Scalability Improvements

Methodologies

• Agile
• Scrum
• SDLC

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 6-8 years of experience in Data Engineering or Business Intelligence.
• Strong hands-on experience with the Incorta platform.
• Expertise in PySpark, Spark SQL, and SQL.
• Experience with ETL processes and enterprise data warehousing.
• Experience integrating data from SAP, Oracle, and enterprise databases.
• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management abilities.

Preferred Qualifications

• Experience working in Big Data environments.
• Knowledge of cloud-based data platforms.
• Experience with Apache Spark ecosystems.
• Exposure to Power BI or Tableau.
• Experience with CI/CD pipelines.
• Familiarity with Python scripting.