Answering Your Kubernetes Storage Questions

Our recent SNIA Cloud Storage Technologies Initiative (CSTI) Kubernetes in the Cloud series generated a lot of interest, but also more than a few questions. The interest is a great indicator of Kubernetes rising profile in the world of computing.

Following the third episode in the series, we’ve chosen a few questions that might help to better explain (or bring additional context to) our presentation. This post is our answer to your very important questions.

If you’re new to this webcast series about running Kubernetes in the cloud, you can catch the three parts here:

The rest of this article includes your top questions from, and our answers to, Part 3:

Q. What databases are best suited to run on Kubernetes?

Read More

Kubernetes Links & Resources to Keep You in the Know

Our recent SNIA CSTI webcast, “Kubernetes in the Cloud (Part 3): (Almost) Everything You Need to Know about Stateful Workloads” offered a wealth of insight on how to address the challenges of running stateful workloads in Kubernetes. This webcast was the third installment of our Kubernetes in the Cloud webcast series and it is now available on-demand as are “Kubernetes in the Cloud (Part 1)” and “Kubernetes in the Cloud (Part 2).”

Our expert presenters, Paul Burt and Ingo Fuchs, have provided additional resources to help keep you in the know on Kubernetes. Here they all are:

Read More

Kubernetes in the Cloud Q&A

Kubernetes is a hot topic these days, generating lots of interest and questions. The goal of our SNIA Cloud Storage Technologies Initiative Kubernetes in the Cloud webcast series is to cut through the hype and provide a vendor neutral look at what Kubernetes is and how it is being used. Our most recent webcast, Kubernetes in the Cloud (Part 2), generated some interesting questions. Here are answers from our expert presenters.

Q. If I’m running my Kubernetes infrastructure at a cloud service provider, do I need CSI support by the cloud provider? If this is not available, I will need a virtual storage array that provides CSI leveraging the underlying cloud storage. Do you know whether there are solutions on the market that I can deploy as a virtual machine at my cloud provider?

A. Current solutions using the CSI interface for public cloud storage are not available at this point. It will be up to the cloud provider to decide whether to support those interfaces to their storage layers.

Q. Does each pod run on one CPU core? I am trying to understand how to size the server configuration?

A. Containers use Linux cgroups to limit the amount of CPU and memory a container can consume and this is exposed in Kubernetes as limits that you can set.

Read More

Stateful Workloads on Kubernetes: (Almost) Everything You Need to Know

Kubernetes is great for running stateless workloads, like web servers. It’ll run health checks, restart containers when they crash, and do all sorts of other wonderful things. So, what about stateful workloads? Large implementers like Uber say to avoid it if you can [1], and gurus like Kelsey Hightower echo that sentiment [2].

It’s the topic we’ll address on August 20th at our live SNIA Cloud Storage Technologies Initiative webcast “Kubernetes in the Cloud (Part 3): Stateful Workloads.”  In this session, we’ll explore when it’s appropriate to run a stateful workload in cluster, or out. We’ll discuss the best options for running a workload like a database on the cloud, or in the cluster, and what’s needed to set that up.

We’ll cover:

Read More

Understanding Kubernetes in the Cloud

Ever wonder why and where you would want to use Kubernetes? You’re not alone, that’s why the SNIA Cloud Storage Technologies Initiative is hosting a live webcast on May 2, 2019 “Kubernetes in the Cloud.”

Kubernetes (k8s) is an open-source system for automating the deployment, scaling, and management of containerized applications. Kubernetes promises simplified management of cloud workloads at scale, whether on-premises, hybrid, or in a public cloud infrastructure, allowing effortless movement of workloads from cloud to cloud. By some reckonings, it is being deployed at a rate several times faster than virtualization.

Read More

Got Questions on Container Storage? We’ve Got Answers!

Keeping up with changes in the world of container storage is not easy. That’s why the SNIA Cloud Storage Technologies Initiative invited expert Keith Hudgins of Docker for a live webcast, “What’s New in Container Storage.” I encourage you to watch it on-demand. It’s well worth the approximately half-hour investment to get up to speed on container storage.

As promised during the live event, here are answers to the questions we received:

Q. How does the new Container Storage Interface fit in here? Read More

Wondering What’s New in Container Storage?

The landscape of containers is moving fast and constantly changing, with new standards emerging every few months. If you wondering what’s new in container storage, you are not alone. That’s why the SNIA Cloud Storage Technologies Initiative is hosting a live webcast on February 26, 2019, “What’s New in Container Storage.”

In this webcast, Keith Hudgins of Docker joins us as a follow up to his earlier container webcast “Intro to Containers, Container Storage and Docker.” It’s our most popular webcast to date with thousands of views. If you missed it, it’s available on demand and will provide you with some great background information before our February 26h webcast.

I encourage you to register today for the February 26th session where you’ll learn:

  • What’s new, what to pay attention to, and how to make sense of the ever-shifting container landscape.
  • Container storage types and Container Frameworks
  • An overview of the various storage APIs for the container landscape
  • How to identify the most important projects to follow in the container world
  • The Container Storage Interface spec and Kubernetes 1.13
  • How to get involved in the container community

It will be live, so bring your questions!

A Q&A on Containers and Persistent Memory

The SNIA Cloud Storage Initiative recently hosted a live webcast “Containers and Persistent Memory.” Where my colleagues and I discussed persistent storage for containers, persistent memory for containers, infrastructure software changes for persistent memory-based containers, and what SNIA is doing to advance persistent memory. If you missed the live event, it’s now available on-demand. You can also download a PDF of the webcast slides.

As promised, we are providing answers to the questions we received during the live event.

Q. How is” Enterprise Server SAN” different from “Traditional” Server SAN?

A. Traditional Server SAN refers to individual servers connected to a dedicated, separate SAN storage solution (e.g. EMC VNX, NetApp FAS, etc.); whereas, Enterprise Server SAN refers to the use of direct-attached-storage that is then aggregated across multiple connected servers to create a “virtual SAN” that is not a separate storage solution, but rather benefits from utilizing the existing capacity contained within the application servers, but in a virtualized, shared pool to improve overall efficiency.

Q. Are there any performance studies done with Containers using Tier 1 apps/Business critical?

A. There have been performance characterizations done on Tier 1, Business Critical applications such as Oracle, MySQL and others. However, this would be vendor specific and the user would have to contact and work with each storage vendor to better understand their specific performance capabilities.

Q. Even though Linux and Microsoft support NVDIMM natively, does the MB/BIOS still need to have support?

A. Yes, the MB needs to have the BIOS enabled to recognize NVDIMMs and it needs the ADR signal wired from the Intel CPU to the DIMMs sockets. The motherboard needs to follow the JEDEC standard for NVDIMMs.

Q. If someone unplugs NVDIMM-N and moves it to another server… what will happen?

A. If the system crashed due to a power loss the data in the NVDIMM will be saved. When it is plugged into another NVDIMM-enabled server the BIOS will check if there is saved data in the NVDIMM and restore that data to DRAM before the system continues to boot.

Q. Are traditional storage products able to support containerized applications?

A. Yes, assuming that they support container orchestration engines such as Docker Swarm or Kubernetes through a “container volume plugin.” However, to the extent that they support containerized applications, it is very specific vendor-to-vendor and there are also a number of new storage products that have been developed exclusively to support containerized applications (e.g. Veritas, Portworx, Robin Systems).

Q. How do the storage requirements for containers compare or differ from those of virtual machines?

A. Actually, “production storage requirements” are very similar—albeit almost equivalent—between containerized applications and applications running within virtual machines; the main difference being that due to the scalability potential of containers, these requirements are often exacerbated. Some of these requirements common to both include: data persistence, data recovery, data performance and data security.

Unlock the Power of Persistent Memory in Containers

Containers and persistent memory are both very hot topics these days. Containers are making it easier for developers to know that their software will run, no matter where it is deployed and no matter what the underlying OS is as both Linux and Windows are now fully supported. Persistent memory, a revolutionary data storage technology, will boost the performance of next-generation packaging of applications and libraries into containers. On July 27th, SNIA is hosting a live webcast “Containers and Persistent Memory.” Read More

Containers, Docker and Storage – An Expert Q&A

Containers continue to be a hot topic today as is evidenced by the more than 2,000 people who have already viewed our SNIA Cloud webcasts, “Intro to Containers, Container Storage and Docker“ and “Containers: Best Practices and Data Management Services.” In this blog, our experts, Keith Hudgins of Docker and Andrew Sullivan of NetApp, address questions from our most recent live event.

Q. What is the major challenge for storage in containerized environment?

A. Containers move fast. Users can spin up and spin down containers extremely quickly. The biggest challenge in production-bound container environments is simply keeping up with the movement of data.

Docker Engine does not delete base container images when the container is shut down. Likewise, Registry assumes you’ve got unlimited storage on hand. For containers that push frequent revisions (as would be the case in a continuous delivery environment), that leads to a lot of orphaned container images that can fill up all available storage if left unchecked.

There are some community-led scripts that will help to keep things in control. That’s the beauty of community-led technology.

Q. What about the speed of retrieving the data from storage?

A. That’s where being a solid storage architect comes in. Every storage system has different strengths and weaknesses, so it’s important to engineer your solution to fit your performance goals. Docker containers are running on the main kernel of the host system. IO is not constrained by abstraction, as in the case of virtual machines. Rather, it is constrained more by density – hundreds of containers on a host can push massive IOPS, so you want your pipes fat and data sources close to the host systems.

Q. Can you expand on moving Docker Volumes from On-Premise bare metal to Cloud Service Providers? Data Migration? Encryption? 

A. None of these capabilities are built-in to Docker Engine. We rely on external storage systems to provide those features. Private-to-cloud replication is primarily a feature of software-based companies, like Portworx, Blockbridge, or Hedvig. Encryption and migration are both common features across other companies as well. Flocker from ClusterHQ is a service broker system that provides many bolt-on features for storage systems they support. You can also use community-supplied services like Ceph to get you there.

Q. Are you familiar with “Flocker” that apparently is able to copy persistent data to another container? Can share your thoughts?

A. Yes. ClusterHQ (makers of Flocker) provide an API broker that sits between storage engines and Docker (and other dynamic infrastructure providers, like OpenStack), and they also provide some bolt-on features like replication and encryption.

Q. Is there any sort of feature in the volume plugins that allows a persistent volume to re-connect to a container if the container is moved across multiple hosts?

A. There’s no feature in plugins to cover that specifically. The plugin API is very simple. In practice, what you would do is write your plugin to expose volumes to Docker Engine on every host that it’s possible to mount that volume. In your container specification, whether it’s a Compose file, DAB file, or what have you, specify the name of your volume. Wherever that unique name is encountered, it will be mounted and attached to the container when it’s re-launched.

If you have more questions on containers, Docker and storage, check out our first Q&A blog: Containers: No Shortage of Interest or Questions.

I also encourage you to join our Containers opt-in email list. It will be a good way to keep up with all the SNIA Cloud is doing on this important technology.