Data Engineer/ Big Data Developer

Spark, Scala, Data Factory, Databrikcs, Azure, ETL, AWS
Contract W2, Contract Independent, Contract Corp-To-Corp, 12 Months
Depends on Experience
Work from home not available Travel not required

Job Description

Position: Data Engineer/Big Data Developer

Location: Houston, TX

Duration: 6-12 Months

1. Developed Spark ETL/ELT jobs that read data from Snowflake and write back to Snowflake
2. Develop scripts Release Management process of Spark job source for with VSTS as the backend to support
3. Develop Azure Data Factory pipelince to create, schedule and manage Spark ETL jobs.
4. Develop and instrament Databricks job code with Azure Application Insights APIs to prform monitoring and alerting
5. Develop Helm Charts to create, install and deply NiFi Azure Kubernetes Services
6. Document, demonstrate and present completed work to dring the end of sprint

In order to so -- The skills required are
1. Good coding skills with preferably SparkSQL and PySpark in addition can be Java and Scala etc.
2. Good knowledge of the many Azure Cloud Services e.g. Azure Databricks, Azure Data Factory, Azure Kubernetes Serices
3. Good knowledge of Spark development with Spark Streaming, SparkSQ, SparkX, Spark ML etc.
5. Exerience with Docker and Kubernetes and IaC and DevOps principals
6. Experence with NiFi, Datameer pipelines and VisualBuilder workbooks
6. Experence with Databricks CLI and VSTS Release Managment APIs
5. Great personality, atitutude to learn and collaborative spirit.
6. Should have successfully delivered good engagement with large enterprise customers here in USA, prefer members of past Devon projects
7. Good intelligence and fresheness of ideas.
8. Experience with Continuous Integration and related tools (i.e. Jenkins, Hudson, Maven, VSTS)
9. Experience with Code Quality Governance related tools (Sonar, Gerrit, PMD, FindBugs, Checkstyle, Emma, Cobertura, etc)
10. Experience with Source Code Management Tools (Github, VSTS)
11. Knowledge of standard tools for optimizing and testing code
12. Ability to operate effectively and independently in a dynamic, fluid environment

The emphasis should be on real-world coding/issue resolution experience and not hello world type of experience.


Posted By

Ashish Lal

750 Astonefield Drive Alpharetta, GA, 30004

Dice Id : 10501470
Position Id : 6187966
Originally Posted : 2 months ago
Have a Job? Post it

Similar Positions

Bigdata Developer (Strong Java)
  • Pyramid Consulting, Inc.
  • Houston, TX
Data Engineer - Houston, TX - Contract-to-hire/Permanent
  • Simplion Technologies Inc
  • Houston, TX
Python Developer
  • Government Systems Technologies Inc. (GSTi)
  • Newark, DE
Big Data
  • Sigma Group
  • Houston, TX
Data Engineer with Cloud
  • CultureFit
  • Houston, TX
Data Engineer
  • DISYS - Digital Intelligence Systems, LLC
  • Houston, TX
Big Data Modeler - Contract - Houston, TX
  • Vaco Technology
  • Houston, TX
Data Engineer
  • Tailored Brands
  • Houston, TX
Data Engineer with Python Exp
  • Pearlsoft Solutions Inc
  • Houston, TX
Data Engineer
  • Procom Services
  • Houston, TX
Data Engineer
  • Houston, TX
BigData Developer / Data Engineer
  • Rivi Consulting Group
  • Houston, TX