Copy-on-Write Links in .NET: Cross-Platform Library
The CopyOnWrite library provides a .NET layer on top of OS-specific logic that provides copy-on-write linking for files (a.k.a. CoW, file cloning, or reflinking). CoW linking provides the ability to copy a file without actually copying the original file's bytes from one disk location to another. The filesystem is in charge of ensuring that if the original file is modified or deleted, the CoW linked files remain unmodified by lazily copying the original file's bytes into each link. Unlike symlinks or hardlinks, writes to CoW links do not write through to the original file, as the filesystem breaks the link and copies in a lazy fashion. This enables scenarios like file caches where a single copy of a file held in a content-addressable or other store is safely linked to many locations in a filesystem with low I/O overhead.
NOTE: In the current package version only Windows functionality is implemented. Issues for Linux and Mac are open and contributions are welcome. Consider reimplementing from the Rust package linked below.
This library allows a .NET developer to:
- Discover whether CoW links are allowed between two filesystem paths,
- Discover whether CoW links are allowed for a directory tree based at a specific root directory,
- Create CoW links,
- Find filesystem CoW link limits.
Discovery is important, as different operating systems and different filesystems available for those operating systems provide varying levels of CoW link support:
- Windows: The default NTFS filesystem does NOT support CoW, but the ReFS filesystem and 2023's new Dev Drive do.
- Linux: Btrfs, Xfs, Zfs support CoW while ext4 does not.
- Mac: AppleFS supports CoW by default.
When using this library you may need to create a wrapper that copies the file if CoW is not available.
Example
using Microsoft.CopyOnWrite;
ICopyOnWriteFilesystem cow = CopyOnWriteFilesystemFactory.GetInstance();
bool canCloneInCurrentDirectory = cow.CopyOnWriteLinkSupportedInDirectoryTree(Environment.CurrentDirectory);
if (canCloneInCurrentDirectory)
{
cow.CloneFile(existingFile, cowLinkFilePath);
}
OS-specific caveats
Windows
File clones on Windows do not actually allocate space on-drive for the clone. This has a good and a possibly bad implication:
- Good: You save space on-disk, as the clones only take up space for region clone metadata (small).
- Possibly bad: If cloned files are opened for append or random-access write, the lazy materialization of the original content into the opened file may result in disk out-of-space errors.
Download Details:
Author: microsoft
Official Github: https://github.com/microsoft/CopyOnWrite
License: MIT
