Data Engineer

Python, Snowflake, data ingestion, data engineering, software engineering, databricks, Scala, integration testing, SQL, terraform, Azure, Apache, Data Processing Frameworks, Hadoop, AWS, Kafka, MongoDB
Full Time
Depends on Experience
Work from home available Travel required to 10%.

Job Description

Title: Senior Data Engineer
Location: Berkeley Heights, NJ & Atlanta, GA, & 100% Remote
Department:Data Management
Compensation: Full-Time Competitive Salary, Health Benefits, 401K
 
Our Company
Axis Group is the Analytics Enablement company. With more than twenty years of experience providing data and analytics consulting services and innovative solutions to enterprises and government organizations, we unleash analytic potential throughout the data lifecycle from data management, governance, visual design and platform migration to digital transformation, predictive analytics, automation and beyond. Axis Group combines business acumen, leadership, and industry-specific experience with technical expertise to tackle tough data problems in order to enable better business outcomes. In addition, we partner with leading data and analytics technologies to deliver the right solutions at the right time. We have experts in: Qlik, ThoughtSpot, Snowflake, Power BI, Dataiku, VoiceBase and StreamSets, among others. We have regional offices in Atlanta, GA and Berkeley Heights, NJ.
 
Senior Data Engineer Responsibilities
  • Develop and automate large scale, high-performance data solutions (batch and/or streaming) to enable Axis Group clients to successfully execute their data strategies
  • Build scalable data pipelines leveraging best-in-class technologies (e.g., StreamSets, Databricks, Confluent) that integrate with high-level orchestration frameworks such as Prefect and Apache Airflow
  • Design and implement multi-tier distributed data platforms in cloud, on-prem and hybrid environments (AWS, Azure, GCP) using infrastructure as code and automation tools (e.g., Terraform, Ansible) as well as container technologies (Docker, Kubernetes, Helm)
  • Develop complex processing logic, custom services and reusable libraries using Python or Java
  • Incorporate best practices from DataOps, MLOps, DevOps and Agile software development into these solutions, including build/deployment automation (CI/CD), testing automation, and operational monitoring/alerting
  • Contribute to shared Data Engineering standards and tooling to establish best practices, to improve productivity, and to increase the quality and repeatability of our solutions
  • Support a wide range of data/analytics use cases across many industries, including cloud data warehousing, streaming, and machine learning
Requirements
  • 5+ years of experience as a Data Engineer or Software Engineer
  • Bachelor's degree in Computer Science or a related field, or equivalent experience
  • Programming expertise in at least one of the following languages: Python, Java, Scala
  • Familiar with the full software development lifecycle including unit testing, integration testing, etc.
  • Production deployment experience to at least one of the following platforms: Snowflake, Databricks, Hadoop, AWS, Azure, GCP
  • Experience with data ingestion tools (e.g., StreamSets, Apache NiFi, Azure Data Factory), data processing frameworks (e.g., Spark, Flink, Beam) and distributed storage (e.g., Kafka, HDFS, S3/ADLS/GCS, MongoDB, Cassandra, Elastic Search) strongly preferred
  • Strong working knowledge of SQL and the ability to write, debug and optimize queries
  • Excellent communication skills, both written and verbal
 
Axis Group is an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected veteran status, age or, any other characteristic protected by applicable law. All candidates must be legally authorized to work in the United States. Axis Group will not consider Visa Sponsorship for this position.
Dice Id : NM14293
Position Id : 6960142
Originally Posted : 9 months ago
Have a Job? Post it