Morioh
Login
Alex Tyler

Alex Tyler

3 years ago

In this blog, I will explain how to run Spark with Kubernetes using the Spark on Kubernetes Operator. I will also describe the configurations for fast S3 data access using S3A Connector and S3A Committers. This architecture works for both cloud object storage and on premise S3 compatible object storage like FlashBlade S3.

#kubernetes #s3 #apache-spark

How to Run Spark with Kubernetes

towardsdatascience.com

How to Run Spark with Kubernetes

Use Spark in a simple and portable way on-promise and in the cloud. In this blog, I will explain how to run Spark with Kubernetes using the Spark on Kubernetes Operator. I will also describe the configurations for fast S3 data access using S3A Connector and S3A Committers. This architecture works for both cloud object storage and on premise S3 compatible object storage like FlashBlade S3.

4.25 GEEK