Big Data Architect

Hybrid in Minneapolis, MN, US • Posted 1 day ago • Updated 1 day ago
Full Time
No Travel Required
On-site
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • data engineering
  • hadoop
  • scala
  • spark
  • azure
  • kubernetes

Summary

Big Data Architect
Minneapolis, MN (3 days onsite)
Fulltime Position
 
 
Required Technical Skills
  • Cloud Platform: Strong hands-on architecture experience with Microsoft Azure, including Azure Data Lake Storage, Azure Databricks, Azure Synapse Analytics, Azure Data Factory, Event Hubs, Azure Functions, Azure Monitor, and Azure security services.
  • Big Data Platforms: Deep expertise in Hadoop ecosystem components such as HDFS, Hive, MapReduce, YARN, Sqoop, Oozie, HBase, and related distributed data processing tools.
  • Distributed Processing: Expert-level experience in Apache Spark, Spark SQL, Spark Structured Streaming, performance tuning, memory management, job optimization, and cluster sizing.
  • Programming: Strong development experience in Scala, along with working knowledge of Python, SQL, shell scripting, and data engineering frameworks.
  • Data Warehousing: Strong Snowflake experience, including warehouse design, schema design, data loading, Snowpipe, performance tuning, clustering, query optimization, cost optimization, and integration with Azure data services.
  • Streaming Technologies: Experience designing and implementing streaming pipelines using Kafka, Azure Event Hubs, Spark Structured Streaming, and event-driven architectures.
  • Storage Optimization: Expertise in Parquet, ORC, Delta Lake, partitioning, bucketing, compaction, compression, indexing, archival strategies, and data lifecycle management.
  • Data Architecture: Experience with data lakes, lakehouse platforms, enterprise data warehouses, data marts, dimensional modeling, metadata management, data lineage, and data governance.
  • DevOps and Automation: Experience with Git, CI/CD pipelines, infrastructure automation, deployment practices, environment management, monitoring, and production operations.
  • Security and Governance: Understanding of role-based access control, encryption, data masking, auditing, secrets management, compliance controls, and enterprise data governance standards.
Required Experience:
  • 15+ years of experience in a strong background in data engineering.
  • 8+ years' experience with Hadoop, Scala, Spark
  • 5+ Years on Azure cloud solutions
 
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10354711
  • Position Id: 9082751
  • Posted 1 day ago
Contact the job poster
MP

Manohar Pediredla

Recruiter @ SRI Tech Solutions
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Minneapolis, Minnesota

18d ago

Easy Apply

Full-time

140 - 145

Minneapolis, Minnesota

12d ago

Easy Apply

Full-time, Third Party

$140000 - $150000

Minnetonka, Minnesota

Today

Full-time

USD 112,700.00 - 193,200.00 per year

Eden Prairie, Minnesota

Today

Full-time

USD 72,800.00 - 130,000.00 per year

Search all similar jobs