We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.
As a Sr. Lead Software Engineer in Assurance Foundations Data Team within Cyber Security and Technology Controls, you will partner with established product, architect and security members of the team and serve as a mentor to newly hired and more junior team members to deliver outcomes for Risk Assurance and strategic stakeholder across the control ecosystem. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.
Job responsibilities
Design, develop, and maintain data ingestion pipelines to collect, process, and transport large volumes of data between various sources, sinks and stores into our data ecosystemCollaborate with data engineers, data scientists, and other stakeholders to normalize, correlate and integrate data from diverse sources, ensuring data quality and consistency.Identify and resolve bottlenecks in data ingestion processes, optimizing performance and throughput.Implement robust monitoring and alerting systems to proactively identify and address data ingestion issues. Troubleshoot and resolve data ingestion problems promptly.Ensure data security and compliance by implementing appropriate access controls, encryption, and data protection measures within the ingestion pipelines.Maintain comprehensive documentation of data ingestion processes, configurations, and best practices.Service Development: Utilize your expertise in a development language (e.g Java, Python) to provide management services for data ingestion solutions, ensuring code quality and performance optimization.Required qualifications, capabilities, and skills
Formal training or certification on data engineering concepts and 5+ years applied experience.Experience deploying, using and monitoring AWS data services including Glue, Athena or NeptuneExperience with Graph Databases, including Cypher and Gremlin, or Relational Databases including DML, DDL, and PL/SQL.Experience managing deployment, upgrade, redeployment and teardown of Cloud services using TerraformExperience integrating data platforms through Continuous Integration & Deployment (CI/CD) - ideally, using git/Bitbucket and Jenkins or spinnakerExperience in building data pipelines using Spark, Glue or similar. Strong understanding of data integration concepts, ETL processes, and data modeling.Proficient in working with various data formats, including JSON, XML, and CSV.Demonstrated execution of full delivery (designing, developing, coding, testing, debugging and documenting) of secure data systems to satisfy business requirements.Preferred qualifications, capabilities, and skills
Experience working in an Agile Development environment and is able to contribute/facilitate Agile ceremonies.Proficient in full stack development having experience creating REST based services on Java/Spring/Spring Boot.Test driven development using modern source control and continuous integrationExperience with API and GraphQL interfaces. Experience with data ingestion, transports and message busses (e.g EKS, Apache Kafka, Apache Nifi, Pentaho, Apache Hop, or similar)Good understanding of Data Warehousing and Data Modelling concepts on AWS Redshift .Knowledge of distributed computing and big data technologies (e.g., Hadoop, Spark) Strong interpersonal and communication skills to work collaboratively in and across teams to achieve common goals