Data Engineer

company banner
GM Financial
Hadoop, Spark and HBase, Kaftka/Scala Coding, Cloudera, Java/Python
Full Time
Depends On Experience
Telecommuting not available Travel not required

Job Description

BASIC FUNCTION 

We are expanding our efforts into complementary data technologies for decision support in areas of ingesting and processing large data sets including data commonly referred to as semi-structured or unstructured data.  Our interests are in enabling data science and search based applications on large and low latent data sets in both a batch and streaming context for processing.

To that end, this role will engage with team counterparts in exploring and deploying technologies for creating data sets using a combination of batch and streaming transformation processes.  These data sets support both off-line and in-line machine learning training and model execution.  Other data sets support search engine based analytics.  Exploration and deployment of technologies activities include identifying opportunities that impact business strategy, collaborating on the selection of data solutions software, and contributing to the identification of hardware requirements based on business requirements.  Responsibility also includes coding, testing, and documentation of new or modified scalable analytic data systems including automation for deployment and monitoring.  This role participates along with team counterparts to develop solutions in an end-to-end framework on a group of core data technologies. 

 

JOB DUTIES 

  • Contribute to the evaluation, research, experimentation efforts with batch and streaming data engineering technologies in a lab to keep pace with industry innovation
  • Work with data engineering related groups to inform on and showcase capabilities of emerging technologies and to enable the adoption of these new technologies and associated techniques
  • Contribute to the definition and refinement of processes and procedures for the data engineering practice
  • Work closely with data scientists, data architects, ETL developers, other IT counterparts, and business partners to identify, capture, collect, and format data from the external sources, internal systems, and the data warehouse to extract features of interest
  • Code, test, deploy, monitor, document, and troubleshoot data engineering processing and associated automation

 

QUALIFICATIONS

Knowledge

  • Excellent knowledge of Linux, AIX, or other Unix flavors

 

Skills

  • Ability to quickly prototype and perform critical analysis and use creative approaches for solving complex problems
  • Excellent written and verbal communication skills

 

Experience

  • 2+ years of hands-on experience with SQL, data modeling, and relational databases such as Oracle, DB2, and Postgres
  • 1+ years of experience with software engineering to include Java, Scala, and Python
  • Experience with processing large data sets with Kafka, RabbitMQ, Flume, Hadoop, HBase, Cassandra and/or Spark or similar distributed system
  • Experience with NoSQL data stores such as MongoDB, Cassandra, HBase, Redis, Riak or other technologies that embed NoSQL with search such as MarkLogic or Lily Enterprise
  • Bachelors or higher degree in computer science or other quantitative discipline or equivalent work experience
  • Experience or familiarity with ETL and Business Intelligence technologies such as Informatica, DataStage, Ab Initio, Cognos, BusinessObjects, or Oracle Business Intelligence 

 

Preferred

  • Experience with populating indexes for search engines such as Solr and Elastic Search
  • Experience with enterprise service bus technologies such as Tibco or Mule
  • Experience with Docker Datacenter, Mesos, Kubernetes, OpenShift and/or Deis or other such container/platform-as-a-service orchestrator
  • Working knowledge of data science and building high performance algorithms

 

WORKING CONDITIONS

 

Work primarily in a controlled climate environment.  Mostly stationary with occasional need to travel between nearby DFW office locations to visit business partner customers and attend meetings.  Occasional travel to attend conferences or training for development pursuits.

 



Company Information

GM Financial, a wholly owned subsidiary of General Motors, is a global provider of auto finance solutions, with operations in the U.S., Canada, China, and Latin America. We employ more than 9,000 hard-working team members in North America, and we're always looking for new people with diverse talents. GM Financial is a workplace where dedicated people have the opportunity to work together and celebrate our successes. Our culture is based on respect, integrity, innovation and personal development.
Dice Id : 1012055SC
Position Id : 401335