Senior Data Engineer

Summary

About Us:

Calsoft is an engineering-led digital product partner for global ISVs and tech-driven enterprises. Since 1998, we’ve helped customers build platforms, modernize systems, and scale with confidence, across cloud, AI, and connected ecosystems. With engineering roots in Silicon Valley and delivery strength in India, we operate at the intersection of product development, intelligent automation, and secure digital transformation.

Senior Data Engineer 
Irvine, CA – Remote

Key Responsibilities 

Data Engineering & Platform Development 

• Design, develop, and maintain scalable batch and streaming data pipelines on AWS.  

• Build robust ETL/ELT frameworks for ingesting data from multiple internal and external sources.  

• Develop reusable data services that support analytics, reporting, machine learning, and operational workloads.  

• Design efficient data models optimized for both analytical and operational use cases.  

• Ensure data quality, consistency, lineage, and governance across the data platform.  

Cloud & Infrastructure 

• Build and optimize cloud-native data infrastructure using AWS services.  

• Design highly available, fault-tolerant, and scalable data processing architectures.  

• Automate deployment, monitoring, and operational workflows using Infrastructure as Code and CI/CD practices.  

• Optimize storage, compute utilization, and overall cloud costs.  

Performance & Reliability 

• Improve pipeline performance, scalability, and reliability.  

• Monitor production workloads and proactively resolve bottlenecks.  

• Optimize SQL queries, data partitioning, indexing strategies, and processing performance.  

• Troubleshoot production issues and implement preventive improvements.  

Collaboration 

• Work closely with Data Scientists, Software Engineers, Product Owners, and Architects to understand business requirements.  

• Support downstream analytics, dashboards, and reporting solutions.  

• Participate in architecture discussions, code reviews, and technical design sessions.  

• Mentor junior engineers and promote engineering best practices.  

Required Technical Skills 

Programming 

• Strong Python programming skills.  

• Experience building production-grade data engineering solutions.  

• Knowledge of object-oriented design, testing, and clean coding practices.  

AWS 

Hands-on experience with several of the following: 

• S3  

• Glue  

• Lambda  

• EMR  

• Athena  

• Redshift  

• RDS  

• Step Functions  

• EventBridge  

• CloudWatch  

• IAM  

• ECS/EKS (preferred)  

Data Engineering 

• ETL/ELT pipeline development  

• Data warehousing concepts  

• Batch and streaming data processing  

• Data modeling  

• Data validation and quality frameworks  

• Metadata and lineage concepts  

Databases 

• Advanced SQL  

• PostgreSQL  

• MySQL  

• Redshift  

• Performance tuning and query optimization  

Development Practices 

• Git  

• CI/CD pipelines  

• Unit testing  

• Docker  

• Agile/Scrum development  

Preferred Skills 

• Apache Spark or PySpark  

• Kafka or Kinesis  

• Airflow or AWS Managed Workflows  

• Infrastructure as Code (Terraform or CloudFormation)  

• Data Lake architecture  

• Lakehouse concepts  

• Experience supporting Machine Learning data pipelines  

• Experience with observability and monitoring tools  

Desired Experience 

• 6–10 years of experience in Data Engineering.  

• Experience building cloud-native data platforms on AWS.  

• Experience handling large-scale structured and semi-structured datasets.  

• Strong understanding of distributed data processing and performance optimization.  

• Experience working in enterprise production environments with high availability requirements. 

Compensation:

Base Annual Compensation:  $125,000 - $180,000

The salary range listed represents the Company’s good-faith estimate of the base compensation reasonably expected to be offered for this position at the time of posting. The actual salary offered to the selected candidate may vary based on job-related factors, including work location, market conditions, relevant experience, education and training, technical skills, demonstrated abilities, internal equity, and the results of role-specific technical evaluations conducted during the selection process. This position may also be eligible for benefits and other forms of compensation, as applicable. Compensation decisions are made in accordance with applicable federal, state, and local laws.

Benefits we offer:

• Comprehensive group health insurance coverage subject to plan terms.
• 401(k) retirement savings plan participation, in accordance with applicable plan provisions.
• Flexible work environment  
• Holidays and paid time off program, subject to company policy.

Calsoft core values:

• Customer Focus
• Integrity
• Innovation
• Ownership
• Excellence
• Team Spirit

Calsoft provides Equal Employment Opportunity to all employees and applicants for employment without regard to race, colour, religion, gender identity or expression, sex, sexual orientation, national origin, age, disability, genetic information, marital status, amnesty, or status as a covered veteran in accordance with applicable federal, state and local laws. Calsoft complies with applicable state and local laws governing non-discrimination in employment.