Job Oppirtunity: Azure Data Engineer: Dallas, TX: C2C

Key Responsibilities:

1. Data Pipeline Development:
   • Design, build, and maintain scalable data pipelines using Azure technologies (Azure Data Factory, Azure Databricks, Azure Synapse Analytics, etc.).
   • Implement ETL (Extract, Transform, Load) processes to collect, clean, and transform raw data from various sources into usable formats.
   • Develop and optimize real-time and batch data processing solutions.
2. Data Storage & Management:
   • Design and manage data storage solutions using Azure data services (Azure Data Lake, Azure Blob Storage, Azure SQL Database, Cosmos DB, etc.).
   • Optimize the performance, scalability, and reliability of data storage systems.
3. Data Integration:
   • Integrate data from different cloud and on-premise sources to ensure data is unified and accessible for business analytics.
   • Work with REST APIs, web services, and other integrations to bring in external data sources into Azure-based systems.
4. Data Warehousing & Analytics:
   • Work with Azure Synapse Analytics (formerly Azure SQL Data Warehouse) to create and manage enterprise-level data warehouses.
   • Build and manage OLAP (Online Analytical Processing) cubes and other reporting systems for high-performance analytics.
5. Security & Compliance:
   • Implement data governance policies and ensure data privacy and security compliance (GDPR, HIPAA, etc.).
   • Manage access control, encryption, and data masking for sensitive data.
6. Collaboration and Communication:
   • Work closely with business analysts, data scientists, and other stakeholders to understand data requirements and deliver high-quality solutions.
   • Provide support for ad-hoc data requests and assist with troubleshooting issues.
7. Monitoring and Optimization:
   • Monitor data pipelines and infrastructure for performance, reliability, and security.
   • Troubleshoot and resolve issues related to data processing, integration, and system performance.
   • Optimize data solutions for cost-efficiency and performance.
8. Documentation and Best Practices:
   • Document data engineering workflows, designs, and procedures for future reference and compliance.
   • Stay up-to-date with the latest trends and best practices in Azure and cloud-based data engineering.

Required Qualifications:

• Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.
• Experience:
  • 6+ years of experience as a Data Engineer, Cloud Engineer, or similar role with a strong focus on Azure technologies.
  • Proven experience with Azure Data Services (Azure Data Factory, Azure Data Lake, Azure Synapse Analytics, Azure SQL Database, Cosmos DB).
  • Experience with ETL tools and data integration techniques.
  • Solid understanding of data warehousing concepts and OLAP systems.
  • Familiarity with programming languages like Python, SQL, or Spark.
• Technical Skills:
  • Proficiency in SQL and database technologies.
  • Experience with data pipeline orchestration tools like Azure Data Factory, Apache Airflow, or similar.
  • Knowledge of big data tools such as Azure Databricks, Apache Spark, Hadoop, and Kafka.
  • Familiarity with cloud security best practices, including identity management and data encryption.
  • Experience with version control systems like Git.
• Soft Skills:
  • Strong problem-solving and troubleshooting abilities.
  • Excellent communication skills to work with cross-functional teams.
  • Ability to manage multiple priorities and meet deadlines in a fast-paced environment.

Preferred Qualifications:

• Certifications:
  • Microsoft Certified: Azure Data Engineer Associate.
  • Microsoft Certified: Azure Solutions Architect Expert.
• Experience:
  • Familiarity with machine learning pipelines and Azure Machine Learning.
  • Experience working in Agile or DevOps environments.
  • Exposure to data governance and master data management concepts.