Overview
Skills
Job Details
Position Title: Databricks developer with IDQ (Databricks certification Must)
Location: Austin, Texas
NON-LOCAL MUST RELOCATE TO AUSTIN TEXAS if not local to Texas
100% remote (initial onboarding and some onsite meetings - candidate must be able to attend in Austin, Texas)
Candidates selected for this opportunity may be required to physically attend meetings in Austin, TX, with minimal notice.
Candidates currently residing outside of the State of Texas may be required to relocate to Texas, if selected for this position and the offer is accepted.
Job Description
The Worker is responsible for developing, maintaining, and optimizing big data solutions using the Databricks Unified Analytics Platform.
This role supports data engineering, machine learning, and analytics initiatives within this organization that relies on large-scale data processing.
Duties include:
- Designing and developing scalable data pipelines
- Implementing ETL/ELT workflows
- Optimizing Spark jobs
- Integrating with Azure Data Factory
- Automating deployments
- Collaborating with cross-functional teams
- Ensuring data quality, governance, and security.
II. CANDIDATE SKILLS AND QUALIFICATIONS
| Minimum Requirements: | ||
| Years | Required/Preferred | Experience |
| 12 | Required | Implement ETL/ELT workflows for both structured and unstructured data |
| 8 | Required | Automate deployments using CI/CD tools |
| 8 | Required | Collaborate with cross-functional teams including data scientists, analysts, and stakeholders |
| 8 | Required | Design and maintain data models, schemas, and database structures to support analytical and operational use cases |
| 8 | Required | Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses |
| 8 | Required | Implement data validation and quality checks to ensure accuracy and consistency |
| 8 | Required | Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging |
| 8 | Required | Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices |
| 8 | Required | Proficiency in Python and R programming languages |
| 8 | Required | Strong SQL querying and data manipulation skills |
| 8 | Required | Experience with Azure cloud platform |
| 8 | Required | Experience with DevOps, CI/CD pipelines, and version control systems |
| 8 | Required | Working in agile, multicultural environments |
| 8 | Required | Strong troubleshooting and debugging capabilities |
| 5 | Required | Design and develop scalable data pipelines using Apache Spark on Databricks |
| 5 | Required | Optimize Spark jobs for performance and cost-efficiency |
| 5 | Required | Integrate Databricks solutions with cloud services (Azure Data Factory) |
| 5 | Required | Ensure data quality, governance, and security using Unity Catalog or Delta Lake |
| 5 | Required | Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL |
| 5 | Required | Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake |
| 1 | Preferred | Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow) |
| 1 | REQUIRED | Databricks Certified Associate Developer for Apache Spark |
| 1 | Preferred | Azure Data Engineer Associate |
Form A :Basic Details of Candidate : ( All Details Mandatory )
- First name:
- Last name:
- Current Location & Zip Code:
- Primary Phone number:
- Alternate Phone number:
- Email:
- Skype ID or MS Teams ID:
- SSN Number (last 4 digits):
- Work Authorisation(with validity):
- LinkedIn ID:
- Passport no:
- Willing to Relocate: Yes / NO:
- D O B(M/D/Y):
- Availability for project:
- Currently Employed:
- Reason for Job Change:
- Best time to take the interview call:
- Any other interviews in pipeline:
- Total Experience:
- Total Relevent Experience:
- USA Experience:
- Bachelor's degree with passing year and university name:
- Master's degree with passing year and university name:
- Available for Criminal background check (Y/N) (required before final hire)
Professional References : (Must Required)
1) Reference Name:
Reference Contact NO:
Reference Email ID :
Reference Designation:
Reference Client working for:
Professional Relationship:
2) Reference Name:
Reference Contact NO:
Reference Email ID :
Reference Designation:
Reference Client working for:
Professional Relationship:
3) Reference Name:
Reference Contact NO:
Reference Email ID :
Reference Designation:
Reference Client working for:
Professional Relationship: