Inline deduplication vs. post-processing deduplication
Will inline or post-processing deduplication work better for your data protection strategy? Learn more about each method with this tip.
In some ways, the more copies of data, the better off a data protection strategy will be. Not every type of copy is necessary, however, and that's where deduplication comes in.
Deduplication is an effective way of conserving storage space by eliminating redundant copies of data. There are two primary forms of deduplication: Inline and post-processing.
There is no universal right or wrong when it comes to inline deduplication vs. post processing. Both have advantages and disadvantages, so the right option for your organization might depend on several factors.
This article will compare inline vs. post-processing deduplication. It will cover how they work, pros and cons of each method, and products on the market today. The article will also explain how both methods of deduplication work with different technologies to protect data and optimize storage space.
What is inline deduplication?
Inline deduplication reduces redundant data in real time, before or while it is sent to the backup device.
Benefits and drawbacks
Inline deduplication is a popular option for primary storage on flash arrays. It reduces the amount of data written to drives, reducing wear on the drives, bandwidth requirements and transfer time. Temporary storage space is not needed for inline deduplication, since copies are reduced before being stored.
However, since the dedupe takes place between servers and backup systems, inline deduplication can cause performance issues during the data backup process.
Products on the market
Several data storage and backup tools use inline deduplication, including the following:
- Veeam Data Platform.
- Dell Avamar and Data Domain.
- IBM Spectrum Protect.
- NEC Hydrastor.
- Quantum DXi.
What is post-processing deduplication?
With post-processing deduplication, the data is backed up to the disk, then reduced. Post-process dedupe is typically scheduled by administrators ahead of time.
Benefits and drawbacks
Post-processing backs up data faster and reduces the backup window. However, this speed requires more disk space to temporarily store backup data, making this method less storage-efficient vs. inline deduplication.
Products on the market
Products that use post-processing deduplication include the following:
- Windows Server.
- HPE ProLiant.
FalconStor, DataCore and Hitachi Vantara offer customers the choice of post-process or inline deduplication.
Deduplication used with other technologies
Both inline deduplication and post-process methods have their advocates, but experts say neither is universally better for data protection -- it all depends on what type of backup environment you have. Many IT teams employ other supplementary techniques and technologies to increase data protection.
Deduplication is often combined with replication for disaster recovery. While deduplication reduces the amount of duplicate data and lowers the bandwidth requirement to copy data off-site, replication copies data from one location to another, providing up-to-date information in the event of a disaster.
Erasure coding, compression and deduplication can work together to protect data and conserve storage capacity, but they have stark differences. Erasure coding enables corrupted data to be reconstructed using information about that data stored elsewhere. Compression reduces the number of bits needed to represent data.
Erin Sullivan is the senior site editor for Informa TechTarget's Data Backup and Disaster Recovery sites.
Dave Raffo is an independent IT analyst and journalist. He previously worked as a senior analyst at The Futurum Group and Evaluator Group, covering integrated systems, software-defined storage, container storage, public cloud storage and as-a-service offerings. He previously worked at TechTarget from 2007 to 2021 as executive news director and editorial director for its storage coverage, and he was a technology journalist for 30 years.
