Backup and Archiving

Backup and archiving are always mentioned together, as both of these technologies support primary data storage. However, the commonalities end over here and cannot be carried forward. But in enterprise IT world it is often observed that archive is analogous to backup.

Simply put, backup and archive are not the same and here’s why explained in simple terms-

Data Backup

Data backup is intended to recover individual lost or corrupt files, or individually corrupt operating system instances. The backed up data has both active and inactive information which encompasses all of your production data. This backup set is useful for purposes of recovery in case of the original copy of data is lost or becomes inaccessible due to reasons. It is always critical that a backup is a copy of production data and the actual data still resides on the production storage systems.

Backups are historically being optimized for large scale recoveries. They are written in large blocks to dedicated hardware like tape libraries or deduplicated disk backup appliances.

On a typical note, these backups are scheduled, often every 24 hours, sometimes more frequently, even hourly with some continuous data protection solutions. The data driven by a backup is stored on a tape or a disk solution or off site like a cloud platform. Restoration from backup can be a complex and lengthy process depending on the volume of data to be restored.

Data archiving

Data archiving on the other hand, is data meant for long term retention, typically for compliance purposes in regulated industries such as finance and legal sectors. These are actually designed with very different access profiles. These systems typically store individual data objects such as files, databases or email messages and usually also capture metadata associated with each item.  The result is that an archive can provide immediate granular access to stored information and so accessing an individual file or email is typically very easy in an archive system.

Generally, archiving solutions which retain and index all copies and versions of a document, file, or email, making them easily are expected to be rapidly retrievable by end users rather than IT admins.

So, please do not treat your backups as archives or vice versa as same, as they serve for different purposes.

Analog CCTV storage

When buying a security DVR system on a strict budget, one of the features you will want to pay special attention to is the amount of storage that comes with your DVR. You’ll want to keep a enough archived history in case you’re out of town or away from your home / business for an extended period of time, but how much storage is enough? Do you want to gamble and keep enough storage for just a handful of days? A week? A month? The longer the time frame, the more storage you’ll need.

Key Factors Affecting the Amount of Storage Space

• # of Days Required
• Quality of Cameras (# of TVL / Megapixels)
• DVR Motion Settings
• DVR Record Rate

For the sake of argument and nice round numbers, we’ll say that a 400 TVL camera takes up 1 MB of memory per minute of recorded footage. By those numbers, a 500GB hard drive would be able to record 512,000 consecutive minutes, or 355.56 days, of completely fictional video footage. Now, let’s say we have a 600 TVL camera that occupies 2.5 MB of memory per minute of recorded footage. That means that the same 500GB hard drive will only be able to record 204,800 consecutive minutes, or 142.22 days, of fake video feeds.

Q: How do I decide how much storage capacity to allow when I'm specifying a digital video recorder?

A: There's no simple answer – every installation must be assessed individually. Key factors affecting storage are picture quality, frame rate, compression method and the length of time for which images are required.

Q: What are the picture quality options?

A: The lowest resolution now normally adopted is CIF (352X288). CIF is generally the rule of thumb when calculating storage capacity, but higher resolutions, such as 2CIF (704X288), 4CIF (704X576) and D1 (720x576), are now often specified. As a guide, CIF images recorded using MPEG4 compression are around 10Kb, 2CIF images around 20Kb and 4CIF around 40Kb. Megapixel cameras usually produce images between 80 and 200Kb each. A balance must be struck between resolution, archive time and budget.

Q: What about frame rates?

A: Always allocate frame rates appropriate to the application. Live motion is 25 full frames-per-second (fps) but each image can be 40Kb or more (4CIF). This means about 1Mb of storage per second of data from each camera – about 3.6Gb per hour. Using 12.5 fps halves storage requirements and still permits lip-sync audio. Where lip sync isn't needed, 4 fps is often acceptable, with corresponding savings in storage.

Q: How do compression methods affect the amount of storage?

A: Significantly! The challenge is to reproduce high quality, high-resolution video using the smallest amount of drive space, but remember that there are no free lunches! If a DVR claims much smaller file sizes than comparable machines with the same compression method, beware – reduced file sizes usually mean reduced quality.

Q: How long should recordings be kept?

A: This depends on the application, but don't automatically adopt the "31-day standard", a hangover from VHS tape. Digital recording is much more flexible. Discuss the options and costs with your client. In general, look at periods where video data cannot be recovered. If this happens to be 20 days, then 20 to 22 days of archive are appropriate.

Q: Is there no easier solution?

A: When in doubt, seek the advice of several professional suppliers to ensure a balanced view. Also, remember that storage is now much less expensive, so over-specifying a little won't significantly affect project costs.

If your DVR has 4SATA/6SATA/8SATA etc then your DVR can take 30days/60Days/90Days/120Days etc. You can used 1TB/2TB/4TB SATA hard Disk for storage. Capture 16-channel DVR Model: DTR4816HD has 8Port SATA.

If we use an ATX Footprint two 9U Rack Enclosures. Each one is Heavy, very nearly 100 Lbs with NOTHING in it. Each Enclosure has 50 Hot Swap Bays for Hard Drives and Two Hard Mount Locations. Today we can put 4.0 Terabyte Hard Drives (SATA-600) in there...up to 100 of them in each of the boxes These drives are SATA Drives with 128 Mb of cache on each one. Additionally, we can add two more hard drives in a hard mount. That gives us up to 400 TB of Storage. As hard drive sizes continue to grow, our storage capability increases. What was literally out of reach for many organizations just a year or so ago, is easy to attain today. You may use 8 Drive, 12 Drive, 16 Drive & 24 Drive Enclosures available.

Extra Note: By Western Digital on Date: 03/12/2014

One major component of every surveillance system is, of course, the cameras. Buyers should opt for kits that offer nothing less than a High Definition (HD) camera that can capture images at a resolution of 1,280 x 720 pixels or even a Full High Definition (FHD) camera that can capture images at 1,920 x 1,080 pixels. High resolution image capture is important as it becomes easier to spot what users are looking for when reviewing the surveillance footage – the last thing users want is for a perpetrator's face (as an example) to be a mess of indistinguishable pixels. And, whether HD or FHD, the cameras should also be able to capture images at a rate of no less than 30 frames per second (FPS). This, again, will help when it comes to reviewing crucial footage.

An equally key component of surveillance systems is the storage being used – in fact this component is what can really make or break the effectiveness of the entire system. If a system is being purchased without storage, the buyer should avoid the temptation to go out and purchase the cheapest hard drive he can find. In most cases this will be some sort of desktop drive that is not designed for 24/7 use, won't be able to capture HD or FHD video from multiple cameras without dropping frames and isn't designed to consume less power and thus generate less heat. This last point is a major concern in terms of reliability, as excessive heat can drastically reduce the life of a hard drive and can also adversely affect read and write operations when the drive is being used in the surveillance system.

A buyer should, ideally, look for hard drives that offer surveillance-relevant optimizations such as AllFrame technology, which not only improves playback performance but works with ATA streaming to reduce errors and frame loss. The drives should also be designed for 24/7 usage and offer features such as IntelliPower, which enables a drive to consume less power and thus generate less heat. This is ideal when a drive is going to be installed in a passively cooled storage enclosure, whether on its own or in-conjunction with several other hard drives.

Considering the high importance of storage in surveillance systems, vendors such as WD have introduced dedicated table-top surveillance drives that boast the aforementioned technology optimizations. The recently introduced WD Purpledrive family has been compatibility tested with hundreds of surveillance systems and offer up to 4TB of capacity on a single drive. Purple drives are uniquely designed for mainstream surveillance systems and offer the perfect blend of performance, reliability and cost - the drives are recommended for use in systems with between 1-to-8 drive bays and where between 1 to as many as 32 HD cameras are used.

Besides selecting the right type of drive, one also has to consider the amount of storage that is needed. Just how much storage is needed varies depending on the specifications of the cameras and then length of time users intend to keep your video surveillance data. The amount of time businesses maintain surveillance data varies drastically but the norm is gradually shifting from as little as 7 days to 30 days. (Certain organizations are legally required to retain data for even longer periods of time.) A general rule of thumb is the longer you are able to retain data, the better it is.

Since the video surveillance enter into megapixel IP era, the required storage capacity significantly increased, which directly increased the demands for hard disc drives.  Western Digital chose to cooperate with Hikvision to launched Purple series HDDs which are dedicated for video surveillance application. Seagate also launched ST4000VX000, which is a surveillance HDD addresses the increasing need for high-resolution cameras and camera counts, and ensures cost-effective performance and durability in always-on surveillance systems. when decide to use HDD, the first thing you want to consider is capacity. You may need to estimate the storage capacity for required video recording. Typically users opt to save costs by simply matching current capacity needs to your video surveillance demands. However, this may actually cost you more and create more upgrade issues in the long run. To choose the HDD capacity, you may take your future needs into consideration. The 4K video resolution is for times higher than your full HD 1080p resolution, which in turn, resulting in requires more than four times the amount of storage space as 1080p. Now, 500GB HDD will disappear from market, the maximum HDD storage capacity can reach up to 6TB, which can provide enough storage capacity for 64 cameras.

Do not use regular desktop computer HDDs for video surveillance. Since the working environment and condition is different, video surveillance requires a HDD that's not susceptible to the issues including heat-related failure or vibration from other drives, which can result in loss of video frames, data loss. What's the difference between regular computer HDD and surveillance dedicated HDD? Compared with computer HDD, surveillance dedicated HDD are designed for 90% write time and only 10% read time. The surveillance-specific drives are also engineered to reliably perform in multi-drive systems with RAID support. Optimized performance and reliability can minimize the effect of vibration from other drives with RV sensors, which mean fewer points for potential failure.

Here we have provided Storage size of Cameras in GB per Day according to their Formats H.264 Compression.

Resolution	Storage Per Day in GB
1 MP	25-30 GB Aprox
2 MP	35-40 GB Aprox
3 MP	60-65 GB Aprox

NAS, DAS, or SAN? - Choosing the Right Storage Technology ?

Data is unquestionably the lifeblood of today's digital organization. Storage solutions remain a top priority in IT budgets precisely because the integrity, availability and protection of data are vital to business productivity and success. But the role of information storage far exceeds day to day functions. Enterprises are also operating in an era of increased uncertainty. IT personnel find themselves assessing and planning for more potential risks than ever before, ranging from acts of terrorism to network security threats. A backup and disaster recovery plan is essential, and information storage solutions provide the basis for its execution.

Businesses are also subject to a new wave of regulatory compliance legislation that directly affects the process of storing, managing and archiving data. This is especially true for the financial services and healthcare industries, which handle highly sensitive information and bear extra responsibility for maintaining data integrity and privacy.

Although the need for storage is evident, it is not always clear which solution is right for your organization. There are a variety of options available, the most prevalent being direct-attached storage (DAS), network-attached storage (NAS) and storage area networks (SAN). Choosing the right storage solution can be as personal and individual a decision as buying a home. There is no one right answer for everyone. Instead, it is important to focus on the specific needs and long-term business goals of your organization. Several key criteria to consider include:
• Capacity - the amount and type of data (file level or block level) that needs to be stored and shared
• Performance - I/O and throughput requirements
• Scalability - Long-term data growth
• Availability and Reliability - how mission-critical are your applications?
• Data protection - Backup and recovery requirements
• IT staff and resources available
• Budget concerns
While one type of storage media is usually sufficient for smaller companies, large enterprises will often have a mixed storage environment, implementing different mediums for specific departments, workgroups and remote offices. In this paper, we will provide an overview of DAS, NAS and SAN to help you determine which solution, or combination of solutions, will best help you achieve your business goals.


DAS: Ideal for Local Data Sharing Requirements

Direct-attached storage, or DAS, is the most basic level of storage, in which storage devices are part of the host computer, as with drives, or directly connected to a single server, as with RAID arrays or tape libraries. Network workstations must therefore access the server in order to connect to the storage device. This is in contrast to networked storage such as NAS and SAN, which are connected to workstations and servers over a network. As the first widely popular storage model, DAS products still comprise a large majority of the installed base of storage systems in today's IT infrastructures. Although the implementation of networked storage is growing at a faster rate than that of direct-attached storage, it is still a viable option by virtue of being simple to deploy and having a lower initial cost when compared to networked storage. When considering DAS, it is important to know what your data availability requirements are. In order for clients on the network to access the storage device in the DAS model, they must be able to access the server it is connected to. If the server is down or experiencing problems, it will have a direct impact on users' ability to store and access data. In addition to storing and retrieving files, the server also bears the load of processing applications such as e-mail and databases. Network bottlenecks and slowdowns in data availability may occur as server bandwidth is consumed by applications, especially if there is a lot of data being shared from workstation to workstation.

DAS is ideal for localized file sharing in environments with a single server or a few servers - for example, small businesses or departments and workgroups that do not need to share information over long distances or across an enterprise. Small companies traditionally utilize DAS for file serving and e-mail, while larger enterprises may leverage DAS in a mixed storage environment that likely includes NAS and SAN. DAS also offers ease of management and administration in this scenario, since it can be managed using the network operating system of the attached server. However, management complexity can escalate quickly with the addition of new servers, since storage for each server must be administered separately.

From an economical perspective, the initial investment in direct-attached storage is cheaper. This is a great benefit for IT managers faced with shrinking budgets, who can quickly add storage capacity without the planning, expense, and greater complexity involved with networked storage. DAS can also serve as an interim solution for those planning to migrate to networked storage in the future. For organizations that anticipate rapid data growth, it is important to keep in mind that DAS is limited in its scalability. From both a cost efficiency and administration perspective, networked storage models are much more suited to high scalability requirements.

Organizations that do eventually transition to networked storage can protect their investment in legacy DAS. One option is to place it on the network via bridge devices, which allows current storage resources to be used in a networked infrastructure without incurring the immediate costs of networked storage. Once the transition is made, DAS can still be used locally to store less critical data.
NAS: File-Level Data Sharing Across the Enterprise

Networked storage was developed to address the challenges inherent in a server- based infrastructure such as direct-attached storage. Network-attached storage, or NAS, is a special purpose device, comprised of both hard disks and management software, which is 100% dedicated to serving files over a network. As discussed earlier, a server has the dual functions of file sharing and application serving in the DAS model, potentially causing network slowdowns. NAS relieves the server of storage and file serving responsibilities, and provides a lot more flexibility in data access by virtue of being independent.

NAS is an ideal choice for organizations looking for a simple and cost-effective way to achieve fast data access for multiple clients at the file level. Implementers of NAS benefit from performance and productivity gains. First popularized as an entry-level or midrange solution, NAS still has its largest install base in the small to medium sized business sector. Yet the hallmarks of NAS - simplicity and value - are equally applicable for the enterprise market. Smaller companies find NAS to be a plug and play solution that is easy to install, deploy and manage, with or without IT staff at hand. Thanks to advances in disk drive technology, they also benefit from a lower cost of entry.

In recent years, NAS has developed more sophisticated functionality, leading to its growing adoption in enterprise departments and workgroups. It is not uncommon for NAS to go head to head with storage area networks in the purchasing decision, or become part of a NAS/SAN convergence scheme. High reliability features such as RAID and hot swappable drives and components are standard even in lower end NAS systems, while midrange offerings provide enterprise data protection features such as replication and mirroring for business continuance. NAS also makes sense for enterprises looking to consolidate their direct-attached storage resources for better utilization. Since resources cannot be shared beyond a single server in DAS, systems may be using as little as half of their full capacity. With NAS, the utilization rate is high since storage is shared across multiple servers.

The perception of value in enterprise IT infrastructures has also shifted over the years. A business and ROI case must be made to justify technology investments. Considering the downsizing of IT budgets in recent years, this is no easy task. NAS is an attractive investment that provides tremendous value, considering that the main alternatives are adding new servers, which is an expensive proposition, or expanding the capacity of existing servers, a long and arduous process that is usually more trouble than it's worth. NAS systems can provide many terabytes of storage in high density form factors, making efficient use of data center space. As the volume of digital information continues to grow, organizations with high scalability requirements will find it much more cost-effective to expand upon NAS than DAS. Multiple NAS systems can also be centrally managed, conserving time and resources.

Another important consideration for a medium sized business or large enterprise is heterogeneous data sharing. With DAS, each server is running its own operating platform, so there is no common storage in an environment that may include a mix of Windows, Mac and Linux workstations. NAS systems can integrate into any environment and serve files across all operating platforms. On the network, a NAS system appears like a native file server to each of its different clients. That means that files are saved on the NAS system, as well as retrieved from the NAS system, in their native file formats. NAS is also based on industry standard network protocols such as TCP/IP, FC and CIFS.

SANs: High Availability for Block-Level Data Transfer

A storage area network, or SAN, is a dedicated, high performance storage network that transfers data between servers and storage devices, separate from the local area network. With their high degree of sophistication, management complexity and cost, SANs are traditionally implemented for mission-critical applications in the enterprise space. In a SAN infrastructure, storage devices such as NAS, DAS, RAID arrays or tape libraries are connected to servers using Fibre Channel. Fibre Channel is a highly reliable, gigabit interconnect technology that enables simultaneous communication among workstations, mainframes, servers, data storage systems and other peripherals. Without the distance and bandwidth limitations of SCSI, Fibre Channel is ideal for moving large volumes of data across long distances quickly and reliably.

In contrast to DAS or NAS, which is optimized for data sharing at the file level, the strength of SANs lies in its ability to move large blocks of data. This is especially important for bandwidth-intensive applications such as database, imaging and transaction processing. The distributed architecture of a SAN also enables it to offer higher levels of performance and availability than any other storage medium today. By dynamically balancing loads across the network, SANs provide fast data transfer while reducing I/O latency and server workload. The benefit is that large numbers of users can simultaneously access data without creating bottlenecks on the local area network and servers.

SANs are the best way to ensure predictable performance and 24x7 data availability and reliability. The importance of this is obvious for companies that conduct business on the web and require high volume transaction processing. Another example would be contractors that are bound to service-level agreements (SLAs) and must maintain certain performance levels when delivering IT services. SANs have built in a wide variety of failover and fault tolerance features to ensure maximum uptime. They also offer excellent scalability for large enterprises that anticipate significant growth in information storage requirements. And unlike direct-attached storage, excess capacity in SANs can be pooled, resulting in a very high utilization of resources. There has been much debate in recent times about choosing SAN or NAS in the purchasing decision, but the truth is that the two technologies can prove quite complementary. Today, SANs are increasingly implemented in conjunction with NAS. With SAN/NAS convergence, companies can consolidate block-level and file-level data on common arrays.

Even with all the benefits of SANs, several factors have slowed their adoption, including cost, management complexity and a lack of standardization. The backbone of a SAN is management software. A large investment is required to design, develop and deploy a SAN, which has limited its market to the enterprise space. A majority of the costs can be attributed to software, considering the complexity that is required to manage such a wide scope of devices. Additionally, a lack of standardization has resulted in interoperability concerns, where products from different hardware and software vendors may not work together as needed. Potential SAN customers are rightfully concerned about investment protection and many may choose to wait until standards become defined.

Conclusion

With such a variety of information storage technologies available, what is the best way to determine which one is right for your organization? DAS, NAS and SAN all offer tremendous benefits, but each is best suited for a particular environment. Consider the nature of your data and applications. How critical and processing-intensive are they? What are your minimum acceptable levels of performance and availability? Is your information sharing environment localized, or must data be distributed across the enterprise? IT professionals must make a comprehensive assessment of current requirements while also keeping long-term business goals in mind.

Like all industries, storage networking is in a constant state of change. It's easy to fall into the trap of choosing the emerging or disruptive storage technology at the time. But the best chance for success comes with choosing a solution that is cost-correct and provides long term investment protection for your organization. Digital assets will only continue to grow in the future. Make sure your storage infrastructure is conducive to cost-effective expansion and scalability. It is also important to implement technologies that are based on open industry standards, which will minimize interoperability concerns as you expand your network.