Azure Data Engineer

Overview

On Site
$40,000 - $60,000
Full Time

Skills

Azure Data Engineer
ADF & Azure Data Brics
ETL
Amazon
Azure Data Lake
SQL
Linux

Job Details

    • Azure Data Services:
    • Azure Data Factory: Automating data movement and transformation using pipelines.
    • Azure Synapse Analytics: Handling large datasets for warehousing and fast querying.
    • Azure Databricks: Leveraging Spark for ETL and machine learning workflows.
    • Azure Data Lake Storage: Storing and managing massive amounts of structured and unstructured data.
    • Azure SQL Database: Managing and querying relational data.
    • Azure Cosmos DB: Working with NoSQL data.
  • Data Modeling and Database Design:
    Understanding how to organize and structure data for optimal performance and scalability.
  • ETL Processes:
    Designing, developing, and implementing Extract, Transform, Load (ETL) pipelines using tools like Azure Data Factory.
  • Big Data Technologies:
    Proficiency in technologies like Hadoop, Spark, and related tools for processing large datasets.
  • Programming Languages:
  • SQL: Essential for managing and querying databases.
  • Python: Used for scripting, data manipulation, and automation.
  • Scala/Java: May be required for more advanced Spark development.
  • Data Security and Compliance:
    Implementing security measures and ensuring compliance with data privacy regulations.
  • Data Governance:
    Establishing policies and procedures for managing data quality, access, and usage.
  • Monitoring and Optimization:
    Monitoring data pipelines and storage for performance and troubleshooting issues
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.