| TxDOT work to be accomplished | "Resource with strong skills in Data architecture and expertise in dimensional data modeling, preferably in a bigdata or an enterprise data warehouse environment. Experience with Informatica Power Exchange for CDC based replication or similar tool is a plus. Expertise in development of data ingestion to Hadoop based data lake (HDFS, Hive) and S3 storage, complex data transformations, and data security. Programming expertise in a major programming/scripting language including but not limited to Java, C++, Scala, python, Go. Hands on delivery capability working in a team across the full lifecycle Data Source Analysis, Design, Development, Testing and Implementation. This includes providing input into requirements, platform development, technical design of the project level technical architecture for ETL, big data application design and development, testing, and deployment of the proposed solution. Proficient in Hadoop ecosystem components such as Hive, Yarn, Knox, Ranger, Sqoop, Oozie, HiveQL, Spark, SQL/data access. Experience in ETL development using Informatica BDM, Kafka, or comparable software. Understanding and skills in data governance, security architecture, load balancing and troubleshooting. Well-versed and deep development experience in Big Data Platform clusters on AWS, HDFS storage optimization, Data Lake/Data Warehouse foundation capabilities.
The Data architect will work with a team of developers, analysts, project managers, and solution architects. It is expected that this person shall have a good understanding of Snowflake cloud data warehouse architectures including loading to dimensional data structures and ETL development for many types of data such as RDBMS sources, API/streaming, structured, and unstructured. Design, develop, support, optimize, and provide technical oversight on all phases of the development life cycle on data warehouse project engagements, including but not limited to participation in data sourcing, analysis, requirements gathering, design, development, testing, deployment, transition, and support. Help monitor and manage the technical scope on engagements to prevent scope creep and deviation from the contract. Follow good practices and project guidelines in all cases including the use of source code control, automated testing and deployment approaches, test-case-based development, and proper documentation practices." |
Never miss an opportunity! Create an alert based on the job you applied for.
