Title: Sr AWS Engineer / Sr AWS Data Engineer (with ETL, Python, AWS Glue, Lambda, Node.js & Microservices exp)
Location: Malvern, PA (Onsite)
Job Type: Contract
Description:
- We are looking for a Senior AWS Engineer to extend our event-driven, serverless financial platform.The platform combines Python-based AWS Glue ETL pipelines with Node.js Lambda microservices.
- A candidate who has built both Python Glue pipelines and Node.js Lambda services in the same production environment would be the strongest fit.
Must Have Skills:
-
AWS Glue ETL
- Microservices - Nodejs or Python
- Deployment pipeline - Jenkins, Cloud formation, SAM
- ETL - Glue Job, Step Functions
- API Gateway - Lamda / DynamoDB
Responsibilities:
Design, build, and maintain independently deployable microservices (single-purpose AWS Lambda functions) within an event-driven architecture
Develop and maintain Node.js (ES6+/async-await) Lambda microservices for file intake, validation, transformation, submission and acknowledgement handling of transaction files (JSON, XLSX, CSV)
Build event-driven Node.js services triggered by S3 event notifications, EventBridge, SQS and API Gateway, with clear contracts between services
Use the AWS SDK for Node.js (v3 preferred) to integrate with core AWS services such as S3, DynamoDB, SNS/SES, Secrets Manager and Step Functions.
Manage dependencies, Lambda layers, bundling and cold start/memory/timeout tuning for Node.js functions
Define clean service boundaries and contracts (event payloads, schemas, S3 formats) enabling independent development and deployment
Maintain and extend AWS Glue ETL jobs (PySpark + native Oracle JDBC) that ingest, transform, and reconcile daily transaction history and position snapshot files
Build and troubleshoot Step Functions workflows (ASL state machines) orchestrating multi-stage pipelines: file validation, ingestion, reconciliation, alerting
Design and maintain Lambda functions (Python and Node.js) for lightweight orchestration, file validation, DynamoDB writes, and SNS/SES alerting
Write and optimize Oracle SQL (native JDBC and Lambda-proxied) against a complex financial schema
Manage DynamoDB data models for transaction history, reconciliation status, and fund-level summaries, including GSIs and TTL-based retention
Implement robust error handling and transactional integrity (per-record commit/rollback, idempotency, duplicate-detection) for financial data where silent failures are unacceptable
Apply retries, dead-letter queues and idempotency keys (for example, a file ID checked with DynamoDB conditional writes) so S3 events delivered more than once never cause duplicate processing
Own unit, integration, and smoke test suites (pytest/unittest, Jest/Mocha, moto and aws-sdk-client-mock for AWS mocking) and maintain/improve code coverage
Support CI/CD pipelines (Jenkins, AWS SAM/CloudFormation), including build, test, lint and packaging of Node.js services
Build and maintain observability: structured CloudWatch logging/metrics, Grafana dashboards, and metric registries for pipeline health
Participate in security reviews (SOC2-aligned findings, SQL injection guards, least-privilege IAM, KMS encryption) and remediate findings, including Node.js dependency vulnerability management (npm audit)
Produce and maintain technical documentation (architecture references, runbooks, deployment guides, changelogs)
Required Skills & Experience:
Python (3.12+) strong proficiency, including testable, well-structured ETL code
Node.js (current LTS) has strong proficiency, including ES6+, async/await, and JavaScript or TypeScript for AWS Lambda microservices
Experience designing and operating microservices architectures: single-purpose services, loose coupling, independent deployment, event contracts and idempotency
AWS Glue (PySpark, Glue 4.0, DynamicFrames) and AWS Lambda (Python and Node.js runtimes)
AWS Step Functions (Amazon States Language) for workflow orchestration
AWS SAM / CloudFormation infrastructure as code, multi-environment deployment
Oracle SQL complex joins, DML across transactional schemas, stored procedure like code patterns
DynamoDB key design, GSIs, conditional writes, pagination
S3 - event-driven architectures (EventBridge notifications), data lake patterns (Parquet)
Strong testing discipline: pytest/unittest, Jest/Mocha, mocking AWS services (moto, aws-sdk-client-mock), stubbing Spark/Glue runtime dependencies for offline unit tests
Experience with financial/transactional data where correctness, auditability, and idempotency are critical
CI/CD experience, ideally Jenkins, with multi-environment promotion gates
Familiarity with CloudWatch metrics/logs, Grafana, and building operational dashboards.