Getting Serious About Containers and Flash Memory

Download this presentation: Getting Serious About Containers and Flash Memory

00:00 Jean Bozman: Thanks very much for tuning in here to Session C-12, getting serious about containers and flash. And it's going to be all about how containers are being delivered and used with flash-based storage systems.

Today, we have a fantastic panel, which I'll introduce you to very shortly, but let me just tell you what this is all about. What we're going to talk about here is the widespread use of containers to speed up distributed applications. As you know, that's changing data centers everywhere. So, what's the effect on storage? Unlike VMs, containers are ephemeral, lasting only a short amount of time. So, storage must be allocated to them rapidly and deallocated as soon as they're no longer around, that's the central thing we're going to look at. Furthermore, standards for just how containers handle storage have been hard to come by, although certainly we're going to talk about there are a few. Both management tools and the flash memory used in systems must be able to handle rapid turnover in a flexible manner to build a solid foundation for public, private and hybrid clouds.

00:57 JB: So, the people we have here on the panel are well suited to discuss this. We can see them on the right, but we have a lovely swag here showing them as well. On the left here, we have Cody Hosterman from Pure Storage, Russ Fellows from the Evaluator Group, Rob Hirschfeld from RackN and Robert Starmer of Kumulus Technologies. Let's just talk a little bit about them first. Russ Fellows of the Evaluator Group is senior partner and analyst for flash and storage, and he leads the firm's Validation and Benchmarking Lab group. Rob Hirschfeld is founder and CEO of RackN, a software firm that automates and provisions cloud infrastructure software for distributed cloud, edge and enterprise data centers. And Cody Hosterman from Pure Storage is technical director of VMware Solution Engineering at Pure Storage. Robert Starmer is founder and CTO of Kumulus Technologies, a DevOps systems reliability engineering and cloud computing consultancy. And I'm Jean Bozman, president of Cloud Architects LLC, and I'm the panel moderator today.

02:03 JB: So, what I've done here is put together some key questions to kick off the discussion, and I expect it to be fairly interactive session. So, let's go to the key questions here. And to open it up, and we're not going to go in an exact order, but if there are some of these questions that really attract your attention panelists, you can tell me that you'd like to jump in with something about that.

First thing I'd like to say is, that I didn't realize how these standards were really coming along, but I've had some discussions with folks about CSI, and of course, I thought CSI was a TV show, but in addition to being a TV show name it's container storage interface. And it's about standardizing access across many types and brands of storage arrays and I didn't realize just how widespread that adoption was. I wonder if someone would jump in and start with that, with CSI and how important it is to getting the containers to work well with various types of storage arrays. Who wants to tackle that one?

03:00 Cody Hosterman: I can jump in first here, I suppose. Yeah, so CSI is one of the subsequent attempts around creating some standardization for storage provisioning and management within an orchestrated container environment. It's not specifically for Kubernetes, but obviously that's generally the most common option when it comes to CSI. There are many different underlying storage platforms -- we have external SAN, we have internal disk, there's software-defined storage offerings, there's a ton of different storage mechanisms that customers might choose, public cloud, on-premises, you name it. And to get things to work at scale and with some consistency from people that may not be storage experts, which is often your developers, your consumers of containers, they want a standardized interface no matter what you choose, where you might move things to. And CSI is the specification that grew from learnings from the traditional FlexVolume offerings and other things that came before that to write a level of consistency within your container environment, so the folks that need the storage don't have to relearn something every time you make an infrastructure change in decision. And I think that's a really key part of what they've done through the CNCF and with the CSI spec itself.

04:04 JB: Right, and how long has this really been a factor? A year, two years? Any idea on that? CSI?

04:11 CH: How many years have we had this year?

[laughter]

04:13 JB: I know. I'm just saying how long have we had this available, so?

04:19 Rob Hirschfeld: Yeah, things move pretty rapidly in the Kubernetes landscape. It was still considered beta this year, right?

04:26 JB: Is that right?

04:28 RH: Yeah, right. So, things move from, "Hey, we just thought of it," to alpha to beta, to people using it in production in less than a year, which is incredibly rapid outside . . .

04:37 JB: It is. It is.

04:40 Robert Starmer: I found it kind of interesting because the technology when I first ran into it was almost two years ago, maybe two and a half years ago, and at the time it was there, people were talking about it like, "This is the answer to storage and storage interactions, and basically providing that vendor-agnostic layer within storage." And yet I was working with a couple of different storage vendors at the time, and they were like, "It's so far from ready, we're not even touching it, we're just working with our own interfaces into this environment." And it was generating all kinds of frustration amongst the customers that were trying to use potentially multiple different storage technologies at the same time, trying to choose the right one for their platform, et cetera. So, it's nice to see it actually come to fruition in the sense that, yes, you now do have a number of vendors that are supporting it. It is a first-class that is not a also-ran component of their storage subsystems.

05:26 JB: Right, and I have a quick comment here too, which is what we're seeing like the story, behind the story behind the story, is that we're seeing so much more movement of the traditional workloads moving into the cloud and they're running into cloud native, which containers are supposed to be there, but you're seeing these two things, the older applications, the newer ones, and they're trying to all be in the same environment. And that's why I think the containerization and digital transformation are just coming together right now, it just seems to be reaching a crescendo in this COVID-19 crisis, financial crisis has only made that acceleration of migration even more quickly. But would one of you want to talk about how you see the rise of adoption of containers for more traditional applications and modernization of older applications? It seems like that's going on in the background.

06:24 RH: I'm happy to start that one, although probably everybody has something to say.

06:27 JB: Yes, everybody, everybody go.

06:30 RH: OK, the reality is containerization solves some critical problems that are applicable for everybody, so even if you don't jump all the way down into a Kubernetes continuous integration cloud-native environment, the packaging and reproducibility with benefits of containers are very real. And at the early days, it was very limited to Linux infrastructure, it was very limited to certain types of environments, we've gotten much, much better about containers being something that everybody could be considering, and so the simple reality is that we are packaging software in containers for many, many applications. And so I don't consider it a cloud-native-only thing because if you look at it actually being cloud native, that's a much heavier lift, it involves decomposing your application into smaller units, being able to handle immutability, injecting credentials and secrets. There's a lot of pieces that go into that, but containers are a good first step and people should really consider it.

07:24 JB: Exactly.

07:25 RS: Well, I think one of the areas where people run into friction with containers is when you start looking at this interaction specific to this conversation we're having here, interaction with storage. Classically, a container is effectively decoupled from a lot of the underlying infrastructure and it also potentially becomes decoupled, or can easily become decoupled, from a connection to the storage that it's using, whether you have some ICS ion clearer or not doesn't actually really impact that, but the normal operations around how you manage the lifecycle of a containerized application and its connectivity to network resources, to storage resources potentially changes. So, while I agree with Rob, absolutely, containers are the right way of thinking about how you bundle and package and even operate your resources, you do have to be aware of those underlying changes and how they're going to impact your operations and especially when it comes to storage. You know, people trying to do databases in a container environment, if they're not aware of those interactions, you get very interesting failures when your data disappears, right?

08:20 JB: Right. I'd be interested to hear what Russ has to say about the adoption of containers for modernization and so on.

08:29 Russ Fellows: Right, so I have a slightly different perspective in that, one of our jobs is to talk to a lot of IT users, right? And I was a former IT administrator, that's where I started out over 30 years ago, IT manager later. So, I have a natural affinity for that and when we talk to our clients, most of them are in more traditional IT roles. So, these are traditional IT administrators. Today, an IT administrator means also, of course, that you're a VM or admin, otherwise you're not a very good IT administrator, and of course, you know, storage administrators and all that. But their perspective is that, "OK these containers are cool and interesting and all the developers are talking about them, great, but how do I manage them?" So, that's the big issue right now for traditional IT people is the management toolset for traditional IT infrastructure managers is incredibly lacking, so if you're not fluent in YAML and love a command line more than anybody else, you're going to have some problems. So, this has not been an easy transition and there's . . . The tools are improving and they're improving rapidly, but still have a long way to go before they're anywhere near like they are for more tradition, you know like a virtual environment for . . .

09:46 JB: Right.

09:47 RF: That's what people want.

09:47 JB: Yeah, I think that's true.

09:48 RF: Like vCenter Server where I can just go in and see all my containers, all my storage, all my networks, at the CSI, CNI, connected all up. That does not exist today.

09:56 JB: Yeah, Russ, it's funny you should mention that, because I was going to ask Cody anyway, something happened over there at VMware very recently, where they had some announcements around vSphere 7 and support for a new environment or toolset called Tanzu and how the VMs were now going to be able to have embedded Kubernetes support and that kind of mitigates in the idea that maybe these two things would coexist in somebody's environment. Cody, do you want to, without going too deep into the product itself, but about the idea of these two ideas being in the same thought, VM containers?

10:36 CH: Yeah, absolutely, because I've been seeing a trend here, right? So, VMware has been doing a lot around their whole product suite around Tanzu and this is a combination of things: Tanzu Kubernetes Grid, which is their essentially managed Kubernetes offering; Tanzu Mission Control, but deploying and managing across all the different clouds, a lot of stuff going along with that and trying to integrate it into the VMware environment. Because Russ is right, a lot of folks that are managing on-premises IT are VMware administrators and they want to be able to enable containers and containerization and container orchestration for their developers, their SREs, whoever's involved in that particular side of it. But, in the end, they don't want to manage two different types of infrastructure. Do I want to start having a practice around their middle? I still have VMs that aren't going anywhere, so I need the ability to run them and generally I want them to be close to a lot of these containers too, because there are application relationships that require a certain amount of adjacency.

11:24 CH: And, so, building a practice around running containers and Kubernetes on top of that VMware environment, alongside of those VMs is attractive for a lot of folks, especially because that's what they're used to, they know how to create VMs, they know about their storage. I mean, VMworld for many years was the de facto storage conference -- right? -- around what storage companies are doing any announcing. And, so, there's a huge ecosystem around backup, around recovery, around resilient storage in VMware environments that folks want to adopt for their container environments because there's a certain zero-trust policy I've seen too here around Kubernetes and containers is that . . . Maybe their application developers are building in resiliency, maybe they are building in some of these tools, maybe they're not. Do central IT want to take that risk, that if there is a disaster, they'll be able to recover on the other side? Do they want to assume that the application developers are doing it? Maybe, maybe not, but who wants to make that their problem, right? And, so, they're kind of making it both sides of it, and so being able to offer these features from a storage and infrastructure level but being able to allow the flexibility of Kubernetes through a VMware environment makes a lot of sense.

12:23 JB: Right, and Rob I wanted you to get into this because we've talked so much in the past about VMs and containers, and you know, how do we manage these as a whole? I'm sure you have some thoughts here about how these two things are going to work together and coexist in an enterprise environment.

12:38 RH: Yeah, I mean today a lot of people are adding containers and therefore Kubernetes in a lot of cases, just on top of the infrastructure they've already built in VMs and so they're . . . We have waves of interest in the idea of bare-metal Kubernetes or bare-metal containers where I can eliminate the virtualization layer attacks, but the Cody's point, people are very used to managing virtual machines. They provide security and storage capabilities and networking capabilities that people know how to operate and are very resilient, and so when you look at container infrastructure, it's solving really a different problem than what the virtual machine infrastructure is solving. And, so, that you really don't operate those things in isolation, and that's what we generally see is that people will still invest in their virtualization infrastructure, well as Russ point, we wait for some maturity and controls to come in behind the scenes. If you were modernizing an application and moving it into containers, you might actually still have parts of that application that reside in traditional virtual machine infrastructure storage.

13:41 RH: And then the goal of the parts you modernize in containers is really decoupled from the underlying infrastructure, and I think this is where we talk about developer versus operator. The beauty of getting this stuff right is that developers are building infrastructure and not really caring, building applications with much less concerns about the infrastructure.

14:00 JB: Right. What's underneath, if you will, not exactly underneath. I wondered if Robert might have some thoughts, too. We were talking earlier about these two worlds kind of coming together and what that's like, just even the developer side and the deployment side, the DevOps piece of this, could you just bring it up a perspective on that?

14:19 RS: Sure, we hear it in the conversation here, right? There's the operator world and people talk about DevOps like somehow operators are going away, and it's all one big happy family that does everything all the time, and certainly there are . . . I would say some small companies, often startup companies that sort of do take a holistic view, willing to do everything soup to nuts. I also talk to lots of little companies where there is a development paradigm and there's an operations paradigm, and in the development paradigm, I don't think there's any question any more, the containers really simplify a lot of the interaction processes, especially when you have multiple people trying to validate and do the same thing on the same set of code, containers really simplify a lot of those operations. You still have to have operations management -- How do you deal with those things over time? How do you provide security for them? All those sorts of things that we would still do within the virtual machine world, it's just different in the container world, you have to think about different aspects of the operations and security, but then when you actually get into operating at scale and looking at some of those interactions, containers are not necessarily a panacea because of . . .

15:19 RS: As I mentioned earlier, this sort of disconnect between the running state of a resource and its actual stateful self because many applications, especially applications that were built pre-cloud-native are not stateless, and stateless is a precursor or a prerequisite for running a cloud-native application. In most cases, they're still state, but state is handled in a different fashion in order to make it truly cloud native and truly stateless in most pieces.

So, that's really the interaction that I think we want to look at, and the converse of that is then how do you actually leverage these new tools to get that benefit that we all look for? How do you actually change your mindset? Not just, "Oh, this is a new tool, and so I can use this tool, but I have to think about how I change my entire development process, my operations process, enhance those processes." Maybe, yes, I do have to learn some command line tools if I want to leverage this today, or I have to find tools like what's now available from VMware and or moving to things like OpenShift or some of the other higher layer tools that are, you know, IBM acquired Red Hat . . .

16:20 JB: And that's had OpenShift.

16:22 RS: Right, they have OpenShift on top of these environments where I change. I'm still using containers that's basically handled for me, but as a developer, I don't see it and as an operator in operating the OpenShift platform that has this containerization component that leverages storage services when appropriate, and has effectively operations built into the platform. So, again, it's a different way of thinking of how these resources can start to tie together to build out a valuable service.

16:50 JB: Right, right, right, right. So, now this next question is kind of a jump ball thing, because we want to bring the container thought closer to the flash storage thought, and it comes back to we need persistent store because the containers are there and then they're being reallocated and so on. I don't know, this might be a question for Russ and Cody. You guys can decide who wants to talk to it first, but how are these two things coming together? We need to have a persistent store and we're using it in a containerized world, anybody? Cody, you want to go at it first and Russ can jump in?

17:23 CH: Yeah, because it's true. There's persistence needed, right? There are databases, folks are starting to containerized databases. Obviously, there are some persistence there, but the important part around the persistence is not just the literal data the application is using, but the metadata within Kubernetes, within the container around the stateful set, understanding how to back that up and recovering it because just recovering the data is not necessarily the only piece. So, CSI while important and cool, it's not the only piece to this puzzle around persistence, and that's where we're seeing a lot of also adoption and investment around backup and recovery, around not just the persistent data sets, what the application metadata itself because that's an important piece of this entire puzzle.

18:00 JB: Well, that's a good point. Russ, do you want to come in here?

18:03 RF: Yeah, I think that's a good point that Cody brought up in that, in general, the container world, there's a lot of benefits to it, but in a lot of ways, it's several years behind more traditional IT. The good news is, for the most part, I see people realizing that and copying the learnings you don't need to relearn how to do rate all over again. That's been done pretty well for 30 years, right? We got in with five implementations, right? So, people are copying appropriately, but like Cody said, data, high availability and recovery availability are paramount for a lot of these applications, no matter how they're run and backup is kind of Step 1 now, but that's not all. You need the replication capabilities. Unfortunately, a lot of people I've found in the container world, when you ask them about replication, they talk about their rate implementation and they think that mirroring is replication. No, that's not really.

19:01 JB: Not really.

[laughter]

19:02 RF: We're talking about how you can recover, so they're just starting to get that concept through, so there's a lot of things that have to happen. I agree, backup is very important, and if you remember back, I think it was about 2005, when people really started using VMs, and you said, "OK, how do you back up your VMs?" Oh, well, you just put an agent and every single VM and backup every VM like it's a standalone host. Yeah, that kind of works, but that was highly inefficient, right? And it took 10 years before VACB came out and was a standard that the backup applications all began using, and changed block tracking, things like that. Like I said, the good news is, the container world realizes they need to do all these things. They're moving very rapidly, they're copying a lot of the good ideas that have already existed. So, I see things progressing quickly, but there certainly isn't parity yet.

19:51 JB: Right, and I'd like to go to Rob now because Rob and I, for a couple years now, been talking about check-in, check-out security . . . What's it called? Directory, having the right container, work with the directory? So, can you just segue here from what we were talking about just now to keeping track of everything and having adequate security and replication? Because one of the disarming things I learned about containers early on from Rob was if I have an insecurity associated with a container, that replication can actually make things worse. So, I think maybe you can speak to that, about how you're just propagating an error into the next container, if you will.

20:38 RH: Be happy to. This is something that I think ties into what Cody and Russ were both talking about. But we haven't said the word yet, which is immutability.

20:47 JB: Ah, you did.

20:48 RH: And so, when people are doing work in containers, a big part of the benefit from a container perspective is that it is an immutable unit of work. So, when you go to deploy a container, unlike . . . We've been . . . A lot of virtual machines, and even servers now, with some of what we do at RackN, have been becoming a single artifact, when you deploy the artifact as a golden image that's been baked into containers from the very start. And, so, when we talk about a containerized system, what we're really doing is we're producing an immutable artifacts. It's only that it doesn't change when it's deployed. That's really relevant to the flash memory storage because that means that everything that happens in that container is designed to be destroyed when the container stops working. And that's core to what things are doing, right? You bring up a container, you run it, you don't change it, you don't configure it, everything is stored in that container while it's running, and then destroyed afterwards. And so, any work that is done must be done outside of the container that you you're distributing from that perspective.

21:48 RH: The place where that gets really important, from security's perspective, is that we're not thinking about traditional patching, we're not thinking about traditional components, which containers are typically built by stacking together a whole bunch of sublayers of that container, which is very efficient, because you don't have to move around containers.

22:07 RH: So, if you have a container system, you have 100 apps, those apps might only use the top layer, everything else, ideally, is going to be standardized and consistent across your whole container infrastructure. But if something injects a security flaw in that, that means that that's going to be replicated very quickly across your whole infrastructure. And so, part of understanding how this is going to work is you have to understand that you need a CI pipeline to build images in a consistent way, right? If you're hand doing it, you're much more in risk -- you could, but I wouldn't recommend it. And then . . . So, you have to have a control process for that, and then you also have to have a way to say, "When I roll out that new image, I have to have a way to get the state, get the secrets, but you don't store them in a container. You have to be able to go and attach it to the information you need, an external database or a file store, an object store, things like that.

22:55 RH: So, containers aren't an isolated technology, they are deeply embedded in the broader system from that perspective, and so you have to think through how you do it, right? If you're putting passwords in a container, you're doing it wrong, pause, learn, do, right? But that's . . . We're used to doing that on server architecture, "Oh, yeah, just have a configuration tool with it. Builds on my configuration files in place and then I don't touch it." That's not containers.

23:21 JB: Wow. So . . .

23:21 RS: Yeah, and containers really do change, they do change things. In the security aspect is one that there's a lot of work that's gone into that, it ties into the storage subsystem, and potentially the replicated storage subsystem, when we think about the fact that, yes, this container is these layers and if my base layers have been secured and encrypted, and basically signed as, yes, it's gone through a validation process. Now at least you're building off of that and you can just be checking the changes that you're making, which means that everything gets faster.

That's one of the things that we don't necessarily talk about. People go to flash because they need high-performance storage. Well, when you're deploying things, when you're migrating things around, if you can leverage this container model of these layers of validated code, then you potentially can still distribute that across your international data centers, even if necessary, and you're still only changing these very small pieces of your system. So, you get the security benefit of being able to check those things, as Rob pointed out, you have to have a CI system, you have to be automating this, and anytime somebody makes a change, all the layers that are changed have to be checked for security problems.

24:23 JB: Right.

24:24 RS: And now you can do that, and it's a lot more performant, too, because it is these layers that you're looking at, not the entire system potentially.

24:30 JB: Yeah. I'm just wondering, listening to what you're saying Robert, is about skill sets. We hear a lot about skill sets, and here we're bringing together several skill sets to make the overall system work. So, actually, I was going to ask Russ, in your work -- and you do a lot of testing work and all -- how many skill sets do users need to have in order to be able to use the container successfully?

24:52 RF: Yeah, that's a great question. A lot. [chuckle]

24:55 JB: Sounds that way.

24:56 RF: Yeah. A lot of environments use a lot of infrastructure-as-code tools, Chef, Puppet, SaltStack, Terraform, Ansible, right? So, these are all common tools, but they're in similar in idea but quite different in their operation and in their language, right? So, just because you know one doesn't mean you can necessarily use another one. You might know the concepts and how they would apply, but it's like learning another programming language, right? So, just because I know Python doesn't mean I can learn Perl just because they both start with Ps. Yeah, they're both programming languages, but they're not the same, right? Concepts are similar, but . . .

25:37 JB: Yeah.

25:38 RF: So, there's a big . . . Yeah, a lot of hurdles to overcome there. Talking, again, about storage, I think there's one thing that's important that we haven't touched on yet. And I think there's, like I said, Kubernetes containers, in general, have used a lot of good ideas from other places, and one of those, I think, is like the dynamic provisioning with persistent volume claims and storage classes. So, you can use a storage class to dynamically create a persistent volume for volume claim. So, that's a great idea.

In a way, it's similar to how VMware does vVols in their storage policy-based management. Who's to say if they copied that idea or not? Doesn't really matter -- it's a good idea, though, in that because when you get hundreds of applications and tens of thousands of storage volumes, trying to manage a volume by some worldwide name is meaningless to anybody. So, you're trying to map some performance of some Fibre Channel volume maybe that you've provisioned and you, the storage end it might know it's Tier 0, super-fast storage class memory volume, but how does the administrator or even farther down the chain, the application developer know that and use that, right?

26:56 RF: So, that's why this idea of having storage classes that can have some meaning like Tier 0 or Tier 1 or something like that makes high-performance storage more consumable, more easily consumable, but also where you don't need that, where you need maybe a backup media, maybe some spinning media, God forbid.

27:18 JB: Hey, it's out there.

27:18 RF: Or QLC, QLC, there's probably a better fit, right? So, who really wants spinning these days? Use QLC.

27:24 JB: Well, it's still out there, that's the thing.

27:27 RF: I know, that's what they tell me. I have a great side story, but not for this . . .

27:32 JB: Yeah, we need a flash memory . . .

27:34 RF: That's right. But, anyway, so I think facilities like that, having the capability to dynamically provision based on policy or storage classes is a great capability that the container world in general, CSI has and I think that's highly applicable to this topic.

27:53 JB: Right.

27:53 RS: And I would also say that the large part of this comes down to education, so we can't just take necessarily the old models and apply them in this new environment because the environment itself is different. The learnings absolutely have to be applied, they have to translate and there's work that's going on in the background to make that easy and happen automatically. But as long as our developers, our operators, learn how these new tools differ, some of the things that we've been talking about here in terms of the storage interactions, for example, I think we'll have much better use and much better value from these new tools. Because as I said before, tools are just that, they're tools, you can use them but if you learn how to use them, you'll actually get better value from those new tools and that's really what we have to be looking at and really enabling for folks.

28:36 JB: Right, right right. Well Cody, I want to call on you because you're from the storage vendor world and you must be looking at a lot of this. Talking to customers who are storage customers and that's what our audience is, largely, what can they do to find their path through this new world of using containers more often and having containers in VMs? Where can they go? What kind of resources can they look to in general to learn more how to take advantage of containers and flash?

29:07 CH: Yeah, I think an important part around this is starting from something that you know and figuring in how it relates to something new. That's how I've generally learned, it's like, "How does this VMware concept relate to this Kubernetes concept?" and what Russ said is perfectly on point because what does VMware CSI driver use to map storage classes to? Storage policies. You create a storage policy within vCenter, you map it and one of the things that VMware's Tanzu offering does is you can create a storage policy right in vCenter. You click on it and say, "Hey, apply this in Tanzu" and so that VMware admin, or that storage admin, or whoever owns those storage policies, they don't need to know how to create YAML, they don't need to know how to run kubectl apply. This is all done from within the VMware layer, and so I think it's about taking the step, the next step to something that you know, so you have a solid foundation to move forward on. And I think storage policies is a really good intersection between learning some of these Kubernetes concepts without having to necessarily dive 100% into something you have no experience in.

30:05 CH: And, so, I think that's a good place to start, understanding how your storage is being used in your environment. That's an important thing, I think, regardless to what you're trying to do and then start digging into how I can offer the benefits that my storage that I manage or rather, the benefits that my storage that I manage offers? How can I provide that to the Kubernetes? How can I add value? Where are the problems around this? And I think there's a lot of great resources, I think a great place to start, learn about CSI. Jean, you know I was talking about this earlier today, there's a Kubernetes podcast and there's and episode on it about CSI -- where it came from and why they built it and why they built it away. Just listening to that is a great place to start.

30:42 JB: Right. I think we're going to have to get to the lightning round in just a minute, but I want to ask [chuckle] I wanted to ask Rob and just for equal time as well, I don't know how much you could talk about OpenShift, but we were talking a lot about other stuff. Do you see, or Russ, do you see IBM and now they acquired Red Hat, is there some kind of angle they're taking on this as well? Because they've got a whole lot of OpenShift containers out there, did you see any particular OpenShift trend with storage?

31:15 RH: Well me, because I want to give everybody a time and so I'll give my brief perspective in that, you know, the concept of OpenShift is a good one in that it's kind of . . . When you go to anybody who wants to know of a possibility to use in the container world go to CNCF.io and look at their landscape page and I did that a few times and after you get completely done being overwhelmed, at one point I think there was like over 100 different Kubernetes offerings, it's like, "Oh, you want Kubernetes. Great, which one of the hundred?" And they're all different in some way, right? So, once you get done, like I said, being overwhelmed, you realize that what you really need is some way to fit all these developers and IT admins especially, they just want something that works. I don't want 100 choices, I just want something that works.

32:02 RH: So, what I think the value of something like OpenShift, and it certainly isn't the only one, is it just says, "Hey, we're going to create a de facto standard. We're just going to take some pieces, put them together and make sure they work." And then developers and administrators at least have a platform that is supported, and somebody at least tried these things out and things sort of worked together so that's a great starting point. So, I think you know, whether OpenShift ends up being the only one -- definitely not -- there's going to be competitors to OpenShift, we've seen that already.

32:29 JB: Oh sure, absolutely.

32:30 RH: But the concept is a good one in that it simplifies interactions for everybody.

32:35 JB: Right. Well, unfortunately, we are going to move to the lightning round and we have to, because we are allowed guys to go a little bit longer because we're one of the last sessions, so don't feel completely constrained. But I think maybe Russ, starting with you, and going around that would be very artful. Just talk about what do you think the kind of key takeaways for this Flash Memory Summit crowd are with respect to containers. And Russ, you could start and then Rob and then Cody and then Robert, if we do it that way. So . . .

33:06 RF: Wandering back to where I started, which is the IT administrators, right? So, beware. Containers are coming, if they haven't already landed in your lap. You're going to have to support them, so start figuring this stuff out. There's going to be some command line initially, unfortunately or fortunately, however you view that. The tools are improving rapidly, but dive into containers and everything that it means, and when you say containers what it really means today is Kubernetes, right?

33:32 JB: Yeah, orchestration level, exactly. Rob, did you want to give your key takeaway for this group of storage professionals?

33:39 RH: Yeah. I think that if you're looking at this, it's not magic. Containers are cgroups and storage networking, and so don't get overwhelmed by what we're talking about. You can decompose containers into base pieces and understand them, so that in itself is not a big deal. The thing that is a little bit hard to wrap around and is worth taking the time is thinking about how it changes the way people build applications and the processes around it, and how we can decouple aspects of the infrastructure. So, if you're used to having a certain role, that will probably change more than how you do that role.

34:14 JB: Ah!

34:15 RH: I think we've really talked about that a bit, but it's worth reinforcing. You don't have to be losing a lot of sleep over understanding the tech, but you might lose some sleep over understanding the digital transformation and the process changes in your organization.

34:28 JB: Right, right. That's excellent, that's excellent. Cody, do you have a key takeaway for the audience?

34:33 CH: Yeah, yeah. I think one is that actually don't be overwhelmed by all these product names and features. Honestly, someone could do the poke and wrap and I wouldn't know if those were Kubernetes features and integrations or they were Pokemon. But the two important things, I think from at least, that I've found recently is that one, the standard things you've learned over your career within infrastructure, within storage, those all apply. The learnings and the wisdom you've gotten from that are absolutely and really important in containerized environments. And a lot of people in that side of the world are new to infrastructure, and so you have a lot to contribute. And I think the other part of this is don't count VMware out just because they have the VM in the name. It's VM on-premises, I think it's VMware's game to lose. They are the incumbent, to a certain extent, to running applications and I think they're doing a pretty good job of keeping there. So, I think in looking at what they're doing and following it is an important piece.

35:22 JB: Yeah, and of course, being an analyst myself, there are a few different ways to approach this, but I think that's an excellent point. Robert, your key takeaway?

35:32 RS: Yeah. I think that the key is there is a new learning that's going to come down the line. As Cody pointed out, what you know already still applies. There are technology changes that are trying to make it easier to apply your current knowledge in a containerized world, but I think the biggest impact really comes from the developer side and developers not understanding that they have changed how they're interacting with storage, and understanding what policies need to reside there and how they need to fit together is really going to be the important shift that everybody needs to make. The developer side, the operator side and the storage teams that are still trying to keep the company's data and really the crown jewels of any company secure. I think education is a key point in making it easy to understand, and it's not always going to just be Kubernetes. I think VMware has a strong suit here. I think tools like OpenShift or Cloud Foundry is another area that changes the game, leveraging this container technology simplifying some of the internal operations and yet we still have to make sure our storage is secure, it's resilient, it's redundant, it's backed up and everything else, all that still has to happen. So, learn everything you can about everything. [chuckle] That's really the answer, I think.

36:37 JB: That's right. OK, so let me say one more thing, and that is, first of all, I want to thank our excellent panel, and it's not over yet, really, because what's going to happen in the conference is that we are going to have a live Q&A session for the people. So, we're actually going to be able to entertain questions in two ways: one, is people can send in questions via chat or they can actually follow-up, at least with me, later on so we can get their questions answered. So, I want to thank everybody. This was terrific.