Lead the architecture, design, and implementation of a scalable big data analytics SaaS platform using Lambda or Kappa architectures to provide data management, governance services, and heterogeneous storage data analytics. Select appropriate AWS (or other cloud) technologies such as S3, Redshift, Kinesis, EMR, Lambda, Glue, Athena, or equivalent to create robust data pipelines. Design real-time streaming and batch processing pipelines, ensuring low-latency, fault-tolerant data processing. Guide the implementation of event-driven, microservices-based architectures that integrate with data storage, compute, and analytics tools. Collaborate with cross-functional teams, including product, data science, and engineering, to understand business requirements and deliver effective technical solutions. Design and implement best practices for data governance, security, and performance optimization. Perform technical evaluations and recommend new technologies and tools for the platform. Lead the transition from a traditional architecture (if applicable) to Lambda/Kappa for big data applications. Ensure the system architecture is robust, scalable, resilient, secure, and efficient, with high availability and disaster recovery built in.

Minimum Requirements

10+ years of experience in software development and data engineering with a focus on architecture roles. Deep expertise in Lambda/Kappa architectures and real-time/batch data processing solutions. Hands-on experience with AWS cloud technologies (S3, Lambda, Kinesis, Glue, Redshift, Athena, EMR, etc.). Similar experience with GCP or Azure is also relevant. Strong knowledge of data lakes, data warehouses, and ETL/ELT pipelines. Proficiency in distributed data processing frameworks like Apache Kafka, Spark, or Flink. Experience with microservices architecture and event-driven design patterns. Excellent knowledge of SQL and NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB). Experience with data governance, security, and compliance frameworks. Experience with DevOps methodologies for deployment, CI/CD, and containerization (Docker, Kubernetes). Strong communication skills and experience leading large-scale, cross-functional projects. Hands-on programming expertise in languages such as Go, Python, C++/Java. Importantly, a programming language should not be a barrier.

Are you passionate about building cutting-edge solutions in the world of big data analytics? Do you thrive in dynamic, fast-paced environments where innovation and collaboration drive success? If you're excited about the opportunity to work on groundbreaking SaaS platforms utilizing modern data architectures like Lambda and Kappa, we want to hear from you!

Join us at Panzura, where you’ll have the chance to design and develop scalable, high-performance systems using the latest cloud technologies and make a real impact on the future of data management and governance. Be a part of a talented team shaping the future of enterprise data services.

The role is for hire through Panzura’s employer of record.