Job Title: Bigdata with Pyspark
Charlotte NC
Contract
Job Summary: We are seeking a highly skilled Senior Data Engineer with a strong background in Big Data technologies, ETL processes, and hands-on experience with AWS. The ideal candidate wit be responsible for designing, developing, and maintaining robust data pipelines and architectures that support our data-driven decision-making processes. This role requires a deep understanding of data engineering principles and the ability to work collaboratively in a fast-paced environment.
Responsibilities:
Design, develop, and implement scalable data pipelines and ETL processes to ingest, process, and store large volumes of data.
Utilize Big Data technologies to manage and analyze complex datasets, ensuring data quality and integrity.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Optimize data storage and retrieval processes to enhance performance and reduce costs.
Implement data governance and security measures to protect sensitive information.
Monitor and troubleshoot data pipeline performance, making necessary adjustments to improve efficiency.
Stay current with industry trends and emerging technologies in data engineering and Bajt Data,
Mandatory Skills:
Strong expertise in Big Data technologies such as Hadoop, Spark, or Kafka.
Hands-on experience with ETL tools and processes.
Proficient in AWS services related to data engineering, including but not limited to S3, Redshift, Glue, and EMR.
Solid programming skills in tanguages such as Python, Java, or Scala.
Experience with data modeling and database design.
Strong analytical and problem-solving skills.
Preferred Skills:
Familiarity with data visualization tools such as Tableau or Power Bl.
Experience with containerization technologies like Docker or Kubernetes.
Knowledge of machine learning concepts and frameworks.
Understanding of data privacy regulations and compliance standards.
Thanks
Vignesh
Never miss an opportunity! Create an alert based on the job you applied for.
