Data Engineer

SOAP, Data science, Performance tuning, Integration, Oracle, Innovation, Apache NiFi, Big data, Collaboration, Continuous integration, Software, MapReduce, HiveQL, Prototyping, Enterprise software, Data marts, Web services, Supervision, Strategic planning, Apache Kafka, Test estimation, Data warehouse, RESTful, Architecture, ETL, Modeling, Agile, scikit-learn, Cloud architecture, Amazon Web Services, DevOps, Python, Apache Hive, Budget, TensorFlow, JSON, Data engineering, Ideation, Apache HBase, ELT, PostgreSQL, Apache Hadoop, Sourcing, MongoDB, Microsoft Windows Azure, NumPy, Jenkins, Apache Spark, Business requirements, Database, QA, Engineering, Troubleshooting, Data QA, SQL, Open source, Data analysis, Test cases, Cloud, XML, Nagios, Flat file, Data architecture, Data processing, Unit testing, Workflow, Data flow, Production support, Data integration, Computer science, Continuous improvement, API, Service level, Productivity, Streaming, Finance, Informatica, Specification, Decision-making, Estimating, Google Cloud, Reporting, MuleSoft, Test-driven development, Analytics
Full Time
$81,000 - $105,000
Work from home not available Travel not required

Job Description

Responsibilities

  • Design, Develop, and unit test new or existing ETL/Data Integration solutions to meet business requirements.
  • Daily production support for Enterprise Data Warehouse including ETL/ELT jobs.
  • Design and Develop data integration/engineering workflows on big data technologies and platforms (Hadoop, Spark, MapReduce, Hive, HBase, MongoDB, Druid)
  • Develop data streams using Apache Spark, Nifi and/or Kafka. Strong Python development for data transfers and extractions (ELT or ETL)
  • Develop workflows in the cloud environment using Cloud base architecture (Azure or AWS)
  • Develop dataflows and processes for the Data Warehouse using SQL (Oracle, Postgres, HIVEQL, SparkSQL & Dataframes)
  • Perform data analysis & model prototyping using Spark/Python/SQL and common data science tools & libraries (e.g. NumPy, Pandas, scikit-learn, TensorFlow)
  • Develop Data integration workflows using Web services in XML, JSON, flat file format, SOAP
  • Participate in troubleshooting and resolving data integration issues such as data quality.
  • Deliver increased productivity and effectiveness through rapid delivery of high-quality applications.
  • Provide work estimates and communicate status of assignments.
  • Assist in QA efforts on tasks by providing input for test cases and supporting test case execution.
  • Analyze transaction errors, troubleshoot issues in the software, develop bug-fixes, involved in performance tuning efforts.
  • Makes some independent decisions and recommendations which affect the section, department and/or division.
  • Participates and provides input to area budget. Works within financial objectives/budget set by management.
  • Develops alternative solutions for decision-making which support organizational goals/objectives and budget constraints.
  • Works with minimum supervision, conferring with superior on unusual matters. Incumbents have considerable freedom to decide on work priorities and procedures to be followed. May include limited supervisory responsibilities.
  • Provide reporting and analytics functionality to monitor API usage and load (overall hits, completed transactions, number of data objects returned, amount of compute time and other internal resources consumed, volume of data transferred).
  • Use results from API reporting/analytics to guide API Developer offering within an organization's overall continuous improvement process and for defining software Service-Level Agreements for APIs.
  • Performs other duties as assigned.

Minimum Qualifications

  • Bachelor’s degree in Computer Science, Engineering, or related field from an accredited university
  • Proven experience in a data integration role with expert level SQL
  • Experience with Cloud base architecture (example: Bluemix, Google Cloud or AWS development)
  • Experience using Apache Spark, Nifi and/or Kafka
  • Experience using Python and Spark/Python/SQL and common data science tools & libraries (e.g. NumPy, Pandas, scikit-learn, TensorFlow)
  • Proven experience integrating enterprise software using ETL modules/Data Engineering tools
  • Knowledge of data architecture, structures and principles with the ability to critique data and system designs
  • Ability to design, create and/or modify data processes that meet key timelines while conforming to predefined specifications utilizing the Informatica and/or Mulesoft platform
  • Experience in big data technologies and platforms (Hadoop, Spark, MapReduce, Hive, HBase, MongoDB)
  • Ability to integrate data from Web services in XML, JSON, flat file format, SOAP
  • Knowledge of core concepts of RESTful API Modeling Language (RAML 1.0) and designing with MuleSoft solutions
  • Experience with Data Science / Data Analyst and their associated tools
  • Experience in Test Driven Development (TDD)
  • Knowledge in DevOps practices and tools (example: Jenkins, Travis CI, UrbanCode Deploy, Nagios)

Posted By

Adrienne Anderson

145 Navarro San Antonio, TX, 78205

Dice Id : RTL136855
Position Id : 6360050
Originally Posted : 3 months ago
Have a Job? Post it

Similar Positions

Data Engineer
  • Tech Quarry
  • San Antonio, TX
Data Engineer (QualityStage)
  • Digital Minds Technologies Inc.
  • San Antonio, TX
Data Quality Lead (Qualitystage)
  • Daman, Inc.
  • San Antonio, TX
SQL Data Analyst
  • Kforce Technology Staffing
  • San Antonio, TX
Business/Data Analyst
  • System One
  • San Antonio, TX
Data Engineer with Hive & SQL
  • Reliable Software Resources
  • San Antonio, TX
Data Quality Lead -Qualitystage
  • nfolks
  • San Antonio, TX
Software Developer
  • Computer Merchant, Ltd., The
  • San Antonio, TX
Data Architect (GRC)
  • Smartek21
  • San Antonio, TX
Senior Data Analyst
  • Vaco Technology
  • San Antonio, TX
Data Centric Testing
  • Wipro Ltd.
  • San Antonio, TX
Support Analyst with experience in Finance and Capital markets
  • Infogium Technologies, LLC
  • New York City, NY
Data Engineer
  • NICE Systems
  • Austin, TX
Senior Data Engineer
  • Robert Half
  • Austin, TX