Tips for streaming files of big data sets
Solid-state drives are a good choice for streaming large video files. Other factors to consider include cost and how often video will be accessed. Read Now
Storing hundreds of terabytes and even petabytes of data is no longer uncommon for organizations throughout a vast array of industries, but there's a big difference between big data sets composed of stagnant archived files and big data containing business-critical streaming media files. These are commonly referred to as big data at rest, and big data in motion.
When determining what type of storage should house a big data set, it's important to take into account how files will be used. While object storage and scale-out network-attached storage (NAS) are two of the most popular storage options for big data environments, media and entertainment companies that rely on data in motion shouldn’t overlook including higher-performance storage, such as solid-state drives (SSDs).
SearchStorage constructed this guide to help storage pros craft an architecture to house big data sets containing large streaming files. From the links provided throughout this guide, learn about the storage challenges you might encounter with big data and how to work around them, as well as the best way to perform analytics. You'll be well on your way to building the best architecture for your big data in motion.
1Exploring big data storage options
Obvious storage choices for many big data platforms include scale-out NAS, for the amount of capacity it has, and object storage, which can help with the unstructured nature of big data sets. But when streaming files, it can also be a good idea to look at high-performance storage, such as SSDs. The stories and podcasts below include examples of what a storage system should provide for big data, and when specific types of storage work the best in big data environments.
2Choosing storage to accommodate big data analytics
Analytics is one of the most important parts of big data, but it can cause a lag in performance. In addition, the type of storage used can affect analytics efficiency. To make big data analytics effective, storage technologies, such as in-memory data grids, and advances in Hadoop continue to evolve. View the links below to learn more about how storage type affects analytics, and which tools can help you gain the most insight from the contents of big data sets.
Effect of big data storage type on analytics
There are two basic types of big data analytics -- synchronous and asynchronous -- but they both have big data storage appetites and specialized needs. Read Now
Processing big data without the performance lag
Low-cost, solid-state memory is powering high-speed analytics of big data streaming from social network feeds and the industrial Internet. Read Now
Using storage to suppress bottlenecks with Hadoop
A variety of performance issues can bog down Hadoop clusters. However, there are ways to sidestep the pitfalls and keep your big data environment humming. Read Now