To get the best performance and reliability from any storage space, strategic storage planning is essential. While much time is spent in equipment planning, just as much effort should be applied towards researching operating systems and their available file systems.

File System Considerations

If you have a choice of file systems, here are some requirements to consider:

==> Volume Size
==> Estimated number of files on the volume
==> Estimated size of files on the volume
==> Velocity
==> Variety
==> Shared volume requirements
==> Backup Requirements

Volume Size

Volume size is an important place to start for planning. This is only the start since strategic planning involves scalability—can it grow as the need arises without interruption of service to the users? The generally accepted truth of filling free space is all too true for data volumes. It is not uncommon to add a terabyte of storage and in six months it’s already half full. Two terabytes (2TB) has become the initial hurdle for many file systems. This limit starts with the SCSI command set being limited to 32-bit logical block addressing. Therefore, a single SCSI LUN using 512 byte block size cannot access over 2TB. File systems that have been used on these systems have been ‘adjusted’ to handle extremely large volumes. However, volumes that are nearing the 2TB limit may be stressing the limits of the file system.

Estimated Number of Files on the Volume

Plan for the number of files that could potentially be stored on the volume. The file system uses Metadata (data about data) to describe the files that are stored. This means there is going to be a certain amount of volume space used by the file system just to manage the files that are there.

File systems that are not built for excessively large directories will slow down applications that access them. This can adversely affect users that have thousands of files on a volume that has millions of files.

Estimated Size of Files on the Volume

Your next consideration should be the sizes of the files that will be on the volume. Organizations that are running large database servers usually have the need to be able to pre- allocate very large files in the gigabyte range of sizes. The file system and operating system need to be able to handle this level of input and output. For these types of enterprises’ systems, expected standard are base on high for performance and integrity.

Velocity

In Big Data terms, velocity refers to how fast data is being produced and how quickly it must be processed to meet business demand and required outcomes. Online transactions, RFID and
retail are just some of the areas that continue to challenge organizations trying to meet these
required outcomes. Cirrus Networks will help you make the most of your data, stored now and in the future. We build and manage sophisticated information management, analytic and flexible, scalable solutions that can underpin your organization operations.

At Cirrus Networks we understand the challenges that data rich environments can present and can help you transform your existing solution into a cost effective modular storage solution. We do this by introducing the latest technologies which intelligently manage data, keep costs down and can adapt to rapidly changing data storage and retrieval requirements.

Variety

A particular challenge in making sense of vast amounts of stored data is how diverse it is. Data can range from numerical, video, email, SMS, audio, photographs, graphics and many other formats. That creates storage and management challenges as the storage and subsequent access characteristics are very different between data types – as is the business insight you can gain from each.

- See more at: http://trendymediablog.com/2014/09/the-ever-growing-challenges-of-data-storage/