Job DescriptionJOB SUMMARY: The
Data Engineer I or II (TO) is responsible for translating data into readily consumable forms to deliver integrated data to consumers by building, operationalizing, and maintaining data pipelines for Data, Analytics & Artificial Intelligence use cases across heterogeneous environments. The
Data Engineer I or II (TO) also plays a role in working with various data integration tools which support a combination of data delivery styles such as virtualization, data replication, messaging and streaming in hybrid and multi-cloud integration scenarios.
JOB REQUIREMENTS: Education/Experience: - Bachelor's degree in computer science (CS), MIS, CIS, Mathematics, Statistics (Theoretical/Computational), Machine Learning or a related field.
- Proven knowledge of data engineering, data integration and data science principles are required
- 6+ years of related work experience in a fast-paced, competitive organization driven by data and enabled by technology
Knowledge/Skills: - Working experience with batch and real-time data processing frameworks.
- Working experience with data modelling, data access, schemas, and data storage techniques.
- Working experience with data quality tools.
- Experience in creating functional and technical designs for data engineering and analytics solutions.
- Experience implementing data models of different schemas and working with diverse data source types.
- Hands-on experience developing solutions with big data technologies such as Hadoop, HIVE and Spark.
- Hands-on experience developing and supporting Python based AI/ML solutions.
- 6+ years hands on experience designing, developing, testing, deploying, and supporting data engineering and analytics solutions using on-premises tools such as, Microsoft's BI Stack (SSIS/SSAS/SSRS), Informatica, Oracle Golden Gate, SQL, Oracle, and SQL Server.
- 4+ years hands on experience designing, developing, testing, deploying, and supporting data engineering and analytics solutions using Microsoft cloud-based tools such as Azure Data Lake, Azure Data Factory, Azure Databricks, Python, Azure Synapse, Azure Key Vault, and Power BI.
- Experience with Containerization methodologies - Docker, OpenShift etc.,
- Experience with Agile as well as DevOps, CI/CD methodologies.
- Hands-on experience designing and developing solutions involving data sourcing, enrichment and delivery using APIs & Web Services.
- Experience working with Jira or similar tools.
- Experience working with Kafka or similar tools.
Job Responsibilities: - Designing and Developing methods to process structured, semi-structured, and unstructured data using batch and real-time data processing techniques.
- Delivering fast, reliable, and scalable data by incrementally and efficiently processing as it arrives from files or streaming sources like Kafka, DBMS and NoSQL.
- Developing Release pipelines to automate recurring manual tasks like creating a build package, checking that build package into a version control repository and deploying it to a DV/UA environment.
- Building and Maintaining templates such as code libraries, pipeline patterns and semantic models to promote reuse and agility.
- Establishing gatekeeping processes that monitor and controls the promotion of successful data processes into production by understanding the business criticality.
- Collaborating with cross-functional teams with a combination of data, business, and technical personas, as well as a product owner/manager as necessary.
- Advocating data reusability by breaking down monolithic data delivery processes into modular data product delivery.
- Ensuring data reliability by defining data quality and integrity controls within the pipeline with defined data expectations and addressing data quality errors with predefined policies.
- Actively working with less experienced data engineers providing technical guidance and oversight.
- Understanding the usage of performance optimization clusters that parallelize jobs and minimize the data movement in Batch and stream data processing.
- Recommending improvements to the processes, technology, and interfaces that reduce the development time and effort and enhance the effectiveness of the team.
- Promptly participating in the Enterprise Social Networking sites, staying up to date on new data technologies and best practices and sharing insights with others in the organization.
Core Behavioral Attributes:Thesuccessful candidate will demonstrate understanding and application of the Southern behaviors:
Unquestionable Trust, Superior Performance, and Total Commitment.
Additional required behavioral attributes:
- Results-oriented
- Innovative
- Strategic thinker with an enterprise view for sustainable solutions
- Committed to continuous learning and improvement
- Committed to the development of others
- Committed to building and maintaining constructive partnerships with business partners
- Works well both independently and with others
- Acts with speed and decisiveness
- Committed to ethical conduct
- Lives and works safely
Tasks are extremely complex and highly technical. The successful person in this position will need to toggle between projects in an emerging and fast-paced area of technology.
About UsAbout Southern Company Southern Company (NYSE: SO) is a leading energy provider serving 9 million customers across the Southeast and beyond through its family of companies. Providing clean, safe, reliable and affordable energy with excellent service is our mission. The company has electric operating companies in three states, natural gas distribution companies in four states, a competitive generation company, a leading distributed energy solutions provider with national capabilities, a fiber optics network and telecommunications services. Through an industry-leading commitment to innovation, resilience and sustainability, we are taking action to meet customers' and communities' needs while advancing our goal of net-zero greenhouse gas emissions by 2050. Our uncompromising values ensure we put the needs of those we serve at the center of everything we do and are the key to our sustained success. We are transforming energy into economic, environmental and social progress for tomorrow. Our corporate culture has been recognized by a variety of organizations, earning the company awards and recognitions that reflect Our Values and dedication to service. To learn more, visit
Southern Company invests in the well-being of its employees and their families through a comprehensive total rewards strategy that includes competitive base salary, annual incentive awards for eligible employees and health, welfare and retirement benefits designed to support physical, financial, and emotional/social well-being. This position may also be eligible for additional compensation, such as an incentive program, with the amount of any bonawards subject to the terms and conditions of the applicable incentive plan(s). A summary of the benefits offered for this position can be found here Additional and specific details about total compensation and benefits will also be provided during the hiring process.
Southern Company is an equal opportunity employer where an applicant's qualifications are considered without regard to race, color, religion, sex, national origin, age, disability, veteran status, genetic information, sexual orientation, gender identity or expression, or any other basis prohibited by law.
About the TeamSouthern Company Services
Never miss an opportunity! Create an alert based on the job you applied for.
