Overview
Remote
Hybrid
$30 - $40
Contract - W2
Contract - 6 Month(s)
No Travel Required
Skills
Python
SQL
Scala
Kafka
GCP
Big Query
Spark
Job Details
Screening Coderbyte over python and SQL
Notes:
- Programming languages: Python (most important), Scala/ Spark
- Cloud: Google Cloud Platform- required
- Message Q: Kafka- required
- Databases: Big Query, SQL, Hive, Map Reduce
- Sponsorship: Can sponsor but does not want to wait for i140. Must be able to convert (will need I40 and need to provide validity date)
- At least 4 years of experience
- Interview process: 30 minutes
Job Duties
- Designing and building ETL pipeline using Sqoop, Hive, Map Reduce and Spark on on-prem and cloud environments;
- Functional programming using Python and Scala for complex data transformations and in-memory computations;
- Using Erwin for Logical/Physical data modeling and Dimensional Data Modeling;
- Designing and developing UNIX/Linux scripts for handing complex file formats and structures
- Orchestration of workflows and jobs using Airflow and Automic;
- Creating multiple Kafka producers and consumers for data transferring;
- Performing Continuous Integration and deployment (CI/CD) using tools like Git, Jenkin to run test cases and build applications with code coverage using Scala test;
- Analyzing data using SQL, Big Query
- Monitoring the cluster performance, setting up alerts, documenting the designs, workflow.
- Providing production support, troubleshooting, and fixing the issues by tracking the status of running applications to perform System administrator tasks.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.