Tag: Storage

A Q&A on Containers and Persistent Memory

Chad Thibodeau Leave a comment

The SNIA Cloud Storage Initiative recently hosted a live webcast “Containers and Persistent Memory.” Where my colleagues and I discussed persistent storage for containers, persistent memory for containers, infrastructure software changes for persistent memory-based containers, and what SNIA is doing to advance persistent memory. If you missed the live event, it’s now available on-demand. You can also download a PDF of the webcast slides.

As promised, we are providing answers to the questions we received during the live event.

Q. How is” Enterprise Server SAN” different from “Traditional” Server SAN?

A. Traditional Server SAN refers to individual servers connected to a dedicated, separate SAN storage solution (e.g. EMC VNX, NetApp FAS, etc.); whereas, Enterprise Server SAN refers to the use of direct-attached-storage that is then aggregated across multiple connected servers to create a “virtual SAN” that is not a separate storage solution, but rather benefits from utilizing the existing capacity contained within the application servers, but in a virtualized, shared pool to improve overall efficiency.

Q. Are there any performance studies done with Containers using Tier 1 apps/Business critical?

A. There have been performance characterizations done on Tier 1, Business Critical applications such as Oracle, MySQL and others. However, this would be vendor specific and the user would have to contact and work with each storage vendor to better understand their specific performance capabilities.

Q. Even though Linux and Microsoft support NVDIMM natively, does the MB/BIOS still need to have support?

A. Yes, the MB needs to have the BIOS enabled to recognize NVDIMMs and it needs the ADR signal wired from the Intel CPU to the DIMMs sockets. The motherboard needs to follow the JEDEC standard for NVDIMMs.

Q. If someone unplugs NVDIMM-N and moves it to another server… what will happen?

A. If the system crashed due to a power loss the data in the NVDIMM will be saved. When it is plugged into another NVDIMM-enabled server the BIOS will check if there is saved data in the NVDIMM and restore that data to DRAM before the system continues to boot.

Q. Are traditional storage products able to support containerized applications?

A. Yes, assuming that they support container orchestration engines such as Docker Swarm or Kubernetes through a “container volume plugin.” However, to the extent that they support containerized applications, it is very specific vendor-to-vendor and there are also a number of new storage products that have been developed exclusively to support containerized applications (e.g. Veritas, Portworx, Robin Systems).

Q. How do the storage requirements for containers compare or differ from those of virtual machines?

A. Actually, “production storage requirements” are very similar—albeit almost equivalent—between containerized applications and applications running within virtual machines; the main difference being that due to the scalability potential of containers, these requirements are often exacerbated. Some of these requirements common to both include: data persistence, data recovery, data performance and data security.

Containers, Docker and Storage – An Expert Q&A

Alex McDonald Leave a comment

Containers continue to be a hot topic today as is evidenced by the more than 2,000 people who have already viewed our SNIA Cloud webcasts, “Intro to Containers, Container Storage and Docker“ and “Containers: Best Practices and Data Management Services.” In this blog, our experts, Keith Hudgins of Docker and Andrew Sullivan of NetApp, address questions from our most recent live event.

Q. What is the major challenge for storage in containerized environment?

A. Containers move fast. Users can spin up and spin down containers extremely quickly. The biggest challenge in production-bound container environments is simply keeping up with the movement of data.

Docker Engine does not delete base container images when the container is shut down. Likewise, Registry assumes you’ve got unlimited storage on hand. For containers that push frequent revisions (as would be the case in a continuous delivery environment), that leads to a lot of orphaned container images that can fill up all available storage if left unchecked.

There are some community-led scripts that will help to keep things in control. That’s the beauty of community-led technology.

Q. What about the speed of retrieving the data from storage?

A. That’s where being a solid storage architect comes in. Every storage system has different strengths and weaknesses, so it’s important to engineer your solution to fit your performance goals. Docker containers are running on the main kernel of the host system. IO is not constrained by abstraction, as in the case of virtual machines. Rather, it is constrained more by density – hundreds of containers on a host can push massive IOPS, so you want your pipes fat and data sources close to the host systems.

Q. Can you expand on moving Docker Volumes from On-Premise bare metal to Cloud Service Providers? Data Migration? Encryption? 

A. None of these capabilities are built-in to Docker Engine. We rely on external storage systems to provide those features. Private-to-cloud replication is primarily a feature of software-based companies, like Portworx, Blockbridge, or Hedvig. Encryption and migration are both common features across other companies as well. Flocker from ClusterHQ is a service broker system that provides many bolt-on features for storage systems they support. You can also use community-supplied services like Ceph to get you there.

Q. Are you familiar with “Flocker” that apparently is able to copy persistent data to another container? Can share your thoughts?

A. Yes. ClusterHQ (makers of Flocker) provide an API broker that sits between storage engines and Docker (and other dynamic infrastructure providers, like OpenStack), and they also provide some bolt-on features like replication and encryption.

Q. Is there any sort of feature in the volume plugins that allows a persistent volume to re-connect to a container if the container is moved across multiple hosts?

A. There’s no feature in plugins to cover that specifically. The plugin API is very simple. In practice, what you would do is write your plugin to expose volumes to Docker Engine on every host that it’s possible to mount that volume. In your container specification, whether it’s a Compose file, DAB file, or what have you, specify the name of your volume. Wherever that unique name is encountered, it will be mounted and attached to the container when it’s re-launched.

If you have more questions on containers, Docker and storage, check out our first Q&A blog: Containers: No Shortage of Interest or Questions.

I also encourage you to join our Containers opt-in email list. It will be a good way to keep up with all the SNIA Cloud is doing on this important technology.

The Next Step for Containers: Best Practices and Data Management Services

Chad Thibodeau Leave a comment

In our first SNIA Cloud webcast on containers, we provided a solid foundation on what containers are, container storage challenges and Docker. If you missed the live event, it’s now available on-demand. I encourage you to check it out, as well as our webcast Q&A blog.

So now that we have set the stage and you’ve become acquainted with basic container technologies and the associated storage challenges in supporting applications running within containers in production, we will be back on December 7th. This time we will take a deeper dive into what differentiates this technology from what you are used to with virtual machines. Containers can both complement virtual machines and also replace them, as they promise the ability to scale exponentially higher. They can easily be ported from one physical server to another or to one platform—such as on-premise—to another—such as public cloud providers like Amazon AWS.

At our December 7th webcast, “Containers: Best Practices and Data Management Services,” we’ll explore container best practices to address the various challenges around networking, security and logging. We’ll also look at what types of applications more easily lend themselves to a microservice architecture versus which applications may require additional investments to refactor/re-architect to take advantage of microservices.

On December 7th, we’ll be on hand to answer your questions on the spot. I encourage you to register today. We hope you can attend!

Open Source Software-Only Storage – Really.

Mario Blandini Leave a comment

Virtually any storage solution is more parts software than hardware. Having said this, users don’t care as much about the percentage of hardware vs. software. They want their consumption experience to be easy and fast to start up, with a pay-as-you-grow model and with the ability to scale without limits. So, it should not be a shock that real IT organizations are using software-only on standard servers to deliver storage to their customers. What’s more, this type of storage can be powered by open source.

At the upcoming SNIA Data Storage Innovation Conference, we are looking forward to discussing software-defined storage (SDS) from a user experience perspective with examples of OpenStack Swift providing an engine for building SDS clusters with any mixed combination of standard server and HDD hardware in a way that is simple enough for any enterprise to dynamically scale.

Swift is a highly available, distributed, scalable object store available as open source.  It is designed to handle non-relational (that is, not just simple row-column data) or unstructured data at large scale with high availability and durability.  For example, it can be used to store files, videos, documents, analytics results, Web content, drawings, voice recordings, images, maps, musical scores, pictures, or multimedia. Organizations can use Swift to store large amounts of data efficiently, safely, and cheaply. It scales horizontally without any single point of failure.  It offers a single multi-tenant storage system for all applications, the ability to use low-cost industry-standard servers and drives, and a rich ecosystem of tools and libraries.  It can serve the needs of any service provider or enterprise working in a cloud environment, regardless of whether the installation is using other OpenStack components.

I know what you are thinking, storage is too critical, so it will never work this way. But the same was said >25 years go when using RAID was seen as too risky given solutions would acknowledge writes while the data was in cache prior to being written to disk. The same was also said >15 years ago when VMware was seen as not robust enough to run any manner of demanding or critical application. Replicas and Erasure Codes are analogous to RAID 1 and RAID 5 respectively, and the uniquely as possible distribution of data behind a single namespace abstracts standard hardware like server virtualization.

Interested in hearing more? Come check out my DSI session, “Swift Use Cases with SwiftStack,” where we look forward to sharing how this new type of storage can work, and to suspend your disbelief that this storage can be enterprise-grade.