Senior Data Engineer

Security represents the most critical priorities for our customers in a world awash in digital threats, regulatory scrutiny, and estate complexity. Microsoft Security aspires to make the world a safer place for all. We want to reshape security and empower every user, customer, and developer with a security cloud that protects them with end to end, simplified solutions. The Microsoft Security organization accelerates Microsoft's mission and bold ambitions to ensure that our company and industry is securing digital technology platforms, devices, and clouds in our customers' heterogeneous environments, as well as ensuring the security of our own internal estate. Our culture is centered on embracing a growth mindset, a theme of inspiring excellence, and encouraging teams and leaders to bring their best each day. In doing so, we create life-changing innovations that impact billions of lives around the world.

In this role you will be a part of the Microsoft Identity security team. This team is on a mission to protect our users from account compromise and fraud. Our data science team is at the core of this mission, creating and improving the machine learning based detections that are used to find and block the malicious actors from harming our systems and customers. As a Data Engineer in the team, you will work closely work with our data scientists and machine learning engineers. You will contribute to implementing the next-gen cloud-based ML and data infrastructure that support a full-stack Data Science team. You will be responsible for creating, maintaining, and optimizing the very large data sets and data pipelines used by our cross-functional Data Science team. You will also partner with the data scientists and analysts to define business metrics and build dashboards to monitor our protections systems. Do you love big data and the challenges it entails? Do you like the idea of applying your technical skills to protect millions of Microsoft customers from malicious actors? Do you want be part of a team of smart, talented and mission driven people? Then this job is for you!

Microsoft's mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

#MSFTSecurity, #IdentityJobs

Responsibilities:

• Design, build, and maintain scalable, reliable, and observable data and ML infrastructure.
• Collaborate closely with data scientists, data source owners, and partner engineering teams to deliver seamless and secure data workflows.
• Create data observability to ensure data health, including data quality and an up-to-date data catalog.
• Optimize performance of big data jobs.
• Participate in technical design and code reviews and contribute to maintaining a clean, secure, and well-documented codebase.
• Proactively identify and resolve bottlenecks and inefficiencies in data pipelines and infrastructure.
• Maintain compliance with all applicable laws and policies across your assigned workstreams.

Other

• Embody our culture and values .

Qualifications:

Required Minimum Qualifications:

Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.

• 3+ years of experience in developing tools and data pipelines processing very large volume data by using Apache Spark and Python or similar tools.
• 3+ years of experience with data governance, data compliance and/or data security.

Other Requirements:

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:

• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.

Additional or Preferred Qualifications:

• Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 8+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.
• 4+ years of experience in data engineering languages like Python, PySpark, SQL and others.
• 3+ years of experience in data modeling.
• 5+ years of hands-on experience with modern data storage and processing technologies, including relational and NoSQL databases, key-value stores, Spark compute engines, distributed file systems such as HDFS and ADLS Gen2, as well as messaging systems like Event Hub.
• Proven expertise in handling and processing large-scale data volumes.
• Experience with GDPR and security compliances.
• Familiarity Azure DevOps, CICD pipeline and release automation.
• Extensive expertise in data observability.

Data Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

;br>
Microsoft will accept applications for the role until August 17, 2025.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form .

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.