Windows Server 2016 has many exciting enhancements, however, one that many will want to take advantage of is the new version of the ReFS file system that is baked into Windows Server 2016. ReFS version 3.1 contains many new enhancements compared to older versions of ReFS that were found in Windows Server 2012. ReFS literally stands for “Resilient File System” as it touts much better built-in resilience than other file systems such as the long standing default NTFS file system. With ReFS being the next generation file system, one would think we would want to use ReFS everywhere. However, we want to think about use cases when it comes to ReFS and when to use Windows Server 2016 ReFs specifically. There are definitely cases to stick with NTFS and ones where we would certainly want to use ReFS.
There are certainly some exciting features included with version 3.1 of ReFS found in Windows Server 2016. Included among those are the following.
Integrity-streams – ReFS uses checksums for metadata and optionally for file data, it can detect corruptions. This includes storage spaces integrations where it can automatically repair detected corruptions.
Data integrity scrubber proactively corrects errors. This periodically scans the volume and identifies corruptions and triggers a repair of the volume.
Real-time tier optimization – On Storage Spaces Direct, this not only tweaks capacity but also delivers high performance to your workloads. This includes creating tiers for your data to live on both for hot data and cold data. Hot data would be data that needs fast storage and cold data that is a capacity tier. So the data gets written to the hot tier first and then moved over to the capacity tier.
Accelerated virtual machine operations. Hyper-V virtualized workloads specifically benefit from ReFS. With block cloning technology built into ReFS 3.1, block data is no longer moved but simply referenced with pointers to blocks. This tremendously increases performance on creating fixed size VHDs based on sparse VDL technology that allows for zeroing out files rapidly. The new block cloning technology enables rapid merge operations for checkpoints in Hyper-V.
Scalability – Scalability improvements with Windows Server 2016 ReFS also allow for extremely large datasets without an impact on performance unlike previous file systems.
When to use Windows Server 2016 ReFS
With the above listed features and benefits from Windows Server 2016 ReFS, when we think about when to use Windows Server 2016 ReFS, there are several use cases.
Virtualization
Virtualization of course here benefits tremendously. With the block cloning technology built-in and the point files being able to reference existing blocks without data moving, this can add tremendous speed for virtual machines and operations with VMs like Hyper-V checkpoints. The quick zeroing out feature with the sparse VDL as well can greatly speed up the time it takes to provision “thick” provisioned eager zero disks.
Backup Solutions
When thinking about the possibilities of the block cloning technology built in, with many of today’s modern backup solutions that perform synthetic full backup operations, they can now utilize the block cloning feature to not move data during these operations but rather place pointer files to reference the blocks needed for the synthetic operation. Companies like Veeam who have touted in their latest version of the Backup & Replication product to take advantage of this technology say they have achieved 10x the speed on a synthetic full backup.
Microsoft Exchange 2016
Microsoft has made mention that Microsoft Exchange Server can take advantage of the enhanced scalability of ReFS. Their comment here:
For example, Exchange and SQL Server AlwaysOn both introduce their own concepts of resiliency and availability suited for their specific workloads, and they can leverage the scalability offered in ReFS.
Microsoft also mentions that SQL Server Always On Groups can also benefit from this scalability aspect of ReFS.
When Not to Use Windows Server 2016 ReFS
While ReFS is certainly an awesome file system with tremendous benefits over NTFS in many areas, there are cases when we don’t want to or simply can’t use ReFS as the file system. These include the following more notable ones.
Boot Drives – You cannot format a boot volume as ReFS. As you can see below from the screenclips of Windows Server 2016 installation, there is no option to format in ReFS. If you click Format you simply get the notice about data loss – no option of file system mentioned.
Deduplication – If you want to use deduplication, you cannot use ReFS. Update Deduplication support has been added to ReFS in Windows Server 2019.
File Encryption – If file level encryption is important to you, ReFS cannot do file encryption. Note you can do BitLocker encryption with ReFS, however, this is whole volume encryption and not at the file level.
Short Names – If you have an application or other use case that you need short file names, this is not supported with ReFS.
Disk Quotas – Disk Quotas are not supported with ReFS. If you need to enforce quotas on a Windows Server 2016 file server you will need to stick with NTFS for now.
Thoughts
When we think about when to use Windows Server 2016 ReFS, there are certainly some really great use cases to the new file system technology. Especially in the areas of virtualization and backup technology, there are huge speed and performance increases by utilizing the block cloning and other technology built into Windows Server 2016 ReFS. However, if you have other needs such as file encryption, short file names, disk quotas and a few others, you need to stick with NTFS for now.
Brandon Lee is the Senior Writer, Engineer and owner at Virtualizationhowto.com, and a 7-time VMware vExpert, with over two decades of experience in Information Technology. Having worked for numerous Fortune 500 companies as well as in various industries, He has extensive experience in various IT segments and is a strong advocate for open source technologies. Brandon holds many industry certifications, loves the outdoors and spending time with family. Also, he goes through the effort of testing and troubleshooting issues, so you don’t have to.
Время на прочтение4 мин
Количество просмотров28K
«ReFS» (Resilient File System) – это новая файловая система от Microsoft, которая создавалась как замена «NTFS». У нее есть несколько солидных преимуществ, а именно, разработчики исправили все ошибки «NTFS». Она гораздо больше защищена от повреждения информации, она лучше переносит возросшую нагрузку, а также масштабируется гораздо проще.
Основные функции Resilient File System
Целостность информации, использование контрольных сумм для метаданных.
Запись информации — Integrity streams (повышенная защита файлов при ошибке части носителя).
«allocate on write» — новая транзакционная модель.
Масштабируемость, увеличенные лимиты на объем каталогов, файлов, разделов.
Работа с пулами разделов, виртуализация разделов.
«data sriping» — система увеличивает производительность и отказоустойчивость данных, избыточная запись информации как в RAID массивах.
Чтобы выявить скрытые ошибки используется функция — «disk scrubbing», очистка диска в фоне.
Пересохранение информации возле проблемных блоков диска.
Единые пулы носителей, на нескольких компьютерах по сети, повышает отказоустойчивость, уменьшает нагрузку.
Поддержка большинства стандартных функций «NTFS».
Система верификации информации.
Отключение «ReFS» невозможно, так как сбойные сектора изолируются.
«Storage Spaces» — новая уникальная гибкая архитектура.
Еще новая ФС унаследовала часть функционала «NTFS»: работа с «BitLocker», «USN» журналирование, «ACL» контролируемый доступ, «mount points»… Естественно, общий объем данных и подключение к разделам«ReFS», доступны по тем же «API».
Особенности «ReFS»
Контрольные суммы теперь используются для метаданных по умолчанию, также их применяют и к данным отдельных файлов. Так, в процессе чтения\записи, осуществляется верификации «на лету». Когда ФС обнаружит повреждение файлов, то моментально удалит записи без перезагрузки компьютера. То есть, «ReFS» теперь самостоятельно себя корректирует при появлении ошибок.
«ReFS» обеспечивает более высокую надежность сохранения информации, по сравнению со старой ФС. Для хранения файлов и метаданных используются «B+-деревья». Размеры, количество разделов и файлов теперь ограничены максимальным 64-битным значением. Пустое пространство хранится в трех разных таблицах, разбитых по объемам фрагментов (малых, средних, больших). Названия файлов и пути пишуться в «Unicode», они не должны превышать 32 килобайта, то есть название файла можно указывать в 30 тысяч знаков.
Защита от отключения питания. Допустим вы прописываете новое имя файла (или другие метаданные), пропало электричество и вы не успели их сохранить. В «NTFS» — файл будет поврежден, так как вы меняете метаданные напрямую. Но «ReFS» всего лишь делает копию метаданных, и не меняет основные пока не произойдет сохранение, особенность работы функции «Copy-on-write».
Технология «Storage Spaces» — это функция виртуализации носителей. Она позволяет создать единое пространство из нескольких физических дисков на одном ПК или нескольких по локальной сети. Также есть возможность настроить «зеркалирование» как RAID массивах.
Отличия от NTFS
«ReFS» изначально создана для поддержки больших объемов разделов, файлов, каталогов и их имен. Новая ФС может включать до двести шестидесяти двух тысяч эксабайт информации, а «NTFS» — только шестнадцать эксабайт.
Еще, в ней отсутствуют функции шифрования, сжатия, дедупликации, дисковые квоты, жесткие ссылки и расширенные атрибуты. Некоторые из них заменены на новые, например, «ReFS» полностью поддерживает шифрование «BitLocker».
Сейчас, в файловую систему «ReFS» вы сможете отформатировать только пул дисков (пространство хранения), где новая ФС покажет себя во всей красе. Но Windows 10 не разрешит отформатировать обычный носитель в «ReFS». Разработчики подчеркивают значение «ReFS» именно для серверов, она доступна на серверных ОС или в «LTSC» версии.
ОС Windows Server 2016 позволит отформатировать обычные тома в «ReFS», но не позволит отформатировать загрузочный диск, потому что загрузочный сектор должен быть на «NTFS» разделе.
Архитектура файловой системы
Структур ReFS значительно отличается от всех остальных файловых систем для Windows. Главными структурными элементами выступают «B+ деревья». Они бывают одноуровневыми (как листья) и многоуровневыми (как деревья). Это обуславливает хорошее масштабирование, для каждого элемента, входящего в структуру ФС. Эта схема, а также 64-битная адресация каждого элемента, делают невозможным проблемы при ее дальнейшем увеличении.
Как корневая запись B+дерева, остальные записи имеют такой же объем в 16 кб, для блока метаданных. Размер в 60 байт — выделен для промежуточных (адресных) узлов. Следовательно, для правильного описания масштабных структур хранения потребуется малое количество уровней. Это позволило увеличить производительность ФС, по сравнению с другими.
Структура файловой системы ReFS
«ReFS» можно определить по специфической сигнатуре, которая расположена в начале раздела:
0x4000 байт — длина всех страниц ReFS.
Номер первой страницы — 0x1e, то есть 0x78000 байт которые идут сразу за загрузочным разделом. Это стандартное отображение Microsoft, которое информирует, что первые метаданные нужно искать после фиксированного смещения.
Алгоритм поиска удаленных данных
Утилиты для восстановления данных выполнят полное сканирование дискового пространства, отформатированного под «ReFS», используя алгоритм анализа по сигнатурам. Проверяя диск блок за блоком, они обнаружат готовые последовательности данных, определят их и выведут результаты. Так как API для работы с дисками для «ReFS» и «NTFS» одинаковы, то и процессы восстановления данных предельно схожи.
Сначала определяется «Volume Header», в нем записано количество секторов на кластер и какой объем сектора. Основная версия лежит в нулевом секторе, а копия расположена в последнем. Далее считывается «Superblock», он расположен в 30-ом блоке и также есть 2 копии во втором и третьем блоке в конце. Из него, извлекается ссылки на «чекпоинт» и его копию, определяется его последняя актуальная версия по «Virtual Allocated Clock».
Checkpoint содержит информацию об основных таблицах, далее считываются заголовки «Page Header» и блоки с указателями (Pointers) на полный список таблиц. Потом ищется «Container Table» для получения физических адресов из виртуальных, и выполняется поиск по «Object ID Table» — все таблицы найдены.
Утилиты доходят до нулевых уровней — то есть «листов b-дерева», и считывают данные файлов. Так как поиск ведется постранично, то если есть сбои — эти элементы просто исключаются из анализа, а сам процесс сканирования идет дальше. Таким образом утилиты для восстановления данных находят всю информацию, которую возможно «достать» с диска.
Полную версию статьи со всеми дополнительными видео уроками смотрите в источнике.
The Resilient File System (ReFS) is Microsoft’s newest file system, designed to maximize data availability, scale efficiently to large data sets across diverse workloads, and provide data integrity by means of resiliency to corruption. It seeks to address an expanding set of storage scenarios and establish a foundation for future innovations.
Key benefits
Resiliency
ReFS introduces new features that can precisely detect corruptions and also fix those corruptions while remaining online, helping provide increased integrity and availability for your data:
- Integrity-streams – ReFS uses checksums for metadata and optionally for file data, giving ReFS the ability to reliably detect corruptions.
- Storage Spaces integration – When used in conjunction with a mirror or parity space, ReFS can automatically repair detected corruptions using the alternate copy of the data provided by Storage Spaces. Repair processes are both localized to the area of corruption and performed online, requiring no volume downtime.
- Salvaging data – If a volume becomes corrupted and an alternate copy of the corrupted data doesn’t exist, ReFS removes the corrupt data from the namespace. ReFS keeps the volume online while it handles most non-correctable corruptions, but there are rare cases that require ReFS to take the volume offline.
- Proactive error correction – In addition to validating data before reads and writes, ReFS introduces a data integrity scanner, known as a scrubber. This scrubber periodically scans the volume, identifying latent corruptions and proactively triggering a repair of corrupt data.
The following features are only available on ReFS:
| Functionality | ReFS | NTFS |
|---|---|---|
| Block clone | Yes | No |
| Sparse VDL | Yes | No |
| Real-time tier optimization | Yes (on Storage Spaces Direct) | No |
The following features are unavailable on ReFS at this time:
| Functionality | ReFS | NTFS |
|---|---|---|
| File system compression | No | Yes |
| File system encryption | No | Yes |
| Data Deduplication | No | Yes |
| Transactions | No | Yes |
| Hard links | No | Yes |
| Object IDs | No | Yes |
| Short names | No | Yes |
| Extended attributes | No | Yes |
| Disk quotas | No | Yes |
| Bootable | No | Yes |
| Supported on removable media | No | Yes |
| NTFS storage tiers | No | Yes |
Skip to content
ReFS in Windows 2016
ReFS, as it is popular known, is a file system first introduced in WS2012 but was less popular due to various limitation. ReFS in WS2016 is vastly improved and focused on Virtualization. ReFS are the file system built on NTFS. Therefore, lots of features offered by NTFS are present in ReFS.
What is Unique about ReFS
Protect Your Data with BDRSuite
ReFS have features which not only can detect data corruption but as well can fix those corruptions. Corruption detection and remediation occurs online. So the biggest advantage is you do not need to take server or volume offline to run chkdisk command.
Corruption detections are achieved using Integrity-Streams. Integrity-Streams by default only does checksums on metadata and optionally also gives you an option to enable checksums on file data.
In ReFS, Proactive error detection & Correction is done by Scrubber. Scrubber periodically scans the volume for checking the metadata and integrity data streams. The scrubber prevents parts of volume getting corrupted by matching the checksums. If checksums do not match, corrupted copies are attempted to be replaced with original copies. In the case of attempts fail Salvaging process starts which remove corrupted file/directory from the namespace of course, without requiring a volume to be taken offline. Scrubbing process is transparent to the user.
ReFS is tightly integrated with Storage Space. Integration with Storage Spaces enables automatic repairing of corrupted data occurs using redundant copies of data. ReFS provide real-time tier optimization by dividing volumes into two logical groups. These two logical groups are referred as tiers can be created based on the disk and resiliency types. Performance and capacity are the two tiers available. All writes occur in performance tier, and eventually, data is moved to capacity tier over a period. In summary, ReFS uses Tiers to write hot data on performance tier and cold data on the capacity tier.
ReFS for Virtualization
Sparse VDL: VDL stands for Valid Data Length. VDL is the actual amount of data written on the disk against the provisioned size. Sparse VDL zeroes out a disk in seconds irrespective of the size of the disk you provision for the VM. Sparse VDL feature is of great use for provisioning VMs as disk allocation happens quickly irrespective of disk size.
Block Cloning: As the term says blocks are cloned. VM Cloning and Checkpoint merging operations make the best use of this feature.
What are the Use Cases of ReFS?
Primarily ReFS is focused on Virtualization. There are major enhancements which makes Hyper-V highly attractive in terms of performance, data integrity. At the same time, it can be used as Backup target for archiving data and finally where NTFS limits the file, volume or directory growth
Virtualization
- The biggest benefits of ReFS can be leveraged by Hyper-V especially features like
Sparse VDL and Block Cloning as explained earlier aids creation, expansion, and merging of VHDX easier - Scalability: ReFS provides features and functionality which efficiently scales and support massive data sets and therefore is considered highly scalable file system. NTFS can create maximum file size of 256 TB, in contrast, ReFS maximum file size is 4.7 Zettabytes which means you can have large volumes, a file which is typically required for hundreds of Virtual Machines
- Real Time Tier operations with Storage Spaces provides high performance as explained above
Data Archival & User Documents
Data integrity of data at rest is very critical. Data in archival is hardly used therefore there is no way to find out if data is corrupted. Scrubbing, Integrity-stream ensures data integrity is intact. Scalability of ReFS file system is also best suited for archived data as the size of data grows in TB’s over a period. It is strongly recommended to use storage spaces to get advantages of ReFS features particularly automatic and online repair. With Storage Spaces integration, redundant data copies help automatic detection and correction of corruption within metadata and data. Scalability of ReFS, therefore, is also applicable to application where data growth is exponential e.g. Exchange and MS SQL Server.
Things to Consider While Planning ReFS
Below things must be considered before deploying ReFS
- You cannot boot from ReFS volume
- You cannot build this file system on removable media
- ReFS does not have below features
- Compression
- Encryption
- Quota management
Many of the best features are available only when used in conjunction with Storage Spaces
Difference Between NTFS and ReFS
In the figure below, it can be seen that Quota tab is not in ReFS volume (right side). Likewise, you do not see compress check box seen on NTFS Volume (left side bottom)
Furthermore, encryption feature available on folder created on NTFS (on the left side) is not available in the folder set up in ReFS which is visible by the absence of advance settings.
Try BDRSuite for Free!
Schedule a live demo with one of our product experts
Start your full-featured 30-day free trial
Explore detailed pricing, editions & features
ReFS is a file system introduced by Microsoft several years ago. It’s important to know about it if you work on the Windows server platform or even with client versions like 10 and 11, as this could improve your experience performing various tasks!
The file format is available in specific versions of Windows 10/11 client operating systems. As you are aware that NTFS (New Technology File System) was introduced almost 26 years ago, and we can clearly say that it is no more a newer file system in the market. Meanwhile, the ReFS (Resilient File System) has advantages and disadvantages in the production environment. In this guide, let us see how to use and format a disk partition in ReFS format (64 kilobytes) in Windows 10/11 and Windows 2019 server versions.
Advantage or Disadvantage?
We will not be looking more at the advantages, disadvantages and tech specifications of this file format. If you want to learn more, you can check this guide.
What I found out is that the ReFS is the best file format if you’re going to store the larger files which rarely get accessed. Especially if you save larger backup files on a disk, it is better to format in ReFS. This file system is most resilient against data corruption compared to NTFS.
Even though it is missing a few features of NTFS such as file-level encryption and compression but if you store larger files (like backup, media, data files), they don’t require encryption. We need better protection against data corruption. Microsoft and other major backup solutions such as Veeam recommend using ReFS file format.
Remember a few key elements of ReFS before you format a disk:
- You can’t install Operating System on the partition.
- Windows App store can’t install apps on this partition.
- Even the installed programs on the partition may not work/open perfectly.
- No file-level compression and encryption (Full disk encryption and BitLocker still work)
As ReFS consumes more computer resources to work, it may impact the lower-level servers and desktop computers. It may not be a significant issue in the data center which usually have high end (right configuration) servers.
Windows Pro and Home Edition do not support the ReFS file format. You will not get an option to format in this file system. Also, a few registry hacks suggested by other websites will not work if you have the wrong version of Windows 10, or 8.1.
You should have Windows 10 or Windows 11 Enterprise to get the ReFS file format. Microsoft has slowly removed ReFS support from Windows 10 Pro with the latest Windows updates.
Right-click on the partition you need to format and select format. Here choose the ReFS file format and unit size. You can leave the unit size to default if you do not have any specific size requirement. Few programs such as backup applications may require the disk in ReFS format with a certain unit size like 64kb, in that case, select the option and format.
You can confirm the file system by checking the properties.
If you want to format in command prompt or change the block
size to a different one (in this case, 4K to 64K), follow the below steps.
Type the below command (Open the command prompt as administrator)
format K: /u /fs:refs /A:64k /i:enable
Here is the partition with a 64K block size. Obviously, this conversion will format the entire disk which leads to data loss.
fsutil fsinfo refsinfo <drive letter:> is a useful command to get the details such as the block size of a ReFS partition. It will be handy when you have the Core version of the Windows server or check through PowerShell.
By the way, if you want to know about partition details from a command prompt, here is another useful command you must know as a server administrator.
fsutil fsinfo volumeinfo <drive letter:>
Format ReFS from Storage Pool
You can get ReFS formatted disk pool from Storage Space in Windows 10. We are not covering it here since you need to have a minimum of two hard disks to create a storage pool which may not be applicable for personal Windows 11/10 computers.
It is easy to create a Storage Pool on the latest Windows system as long as you have more than 2 physical disks. Do not add/modify OS disk. It should be done with the additional 2 more hard disks to create a storage pool. If you want to test it without extra physical hard disks, you can create VHD (Virtual hard disk files) on your existing partition and see how it works (like I created the last 10GB disk in the above example)
Create Partitions in ReFS file format on Windows 2019/2022 Servers
The steps mentioned earlier for Windows 10 workstations are suitable for server versions (like 2012R2 and 2016) s also.
You can format a disk in ReFS format on most of the Windows servers easily without any registry hacks.
The CMD and PowerShell commands work well on the server OS in case you need to use them on Server Core or without desktop experience versions.
Generally, Servers have more disks such as Physical disks, virtual disk files and NAS storage (iSCSI and Fiber disks). Therefore creating a storage pool (space) and formatting it with ReFS gives extra advantages of expanding the disk space whenever wanted in the pool.
Hope this guide gives an overview of the ReFS file system in Windows 10/11 and server versions 2012R2, 2016/2019 and the latest 2022. Also, how to format the disks in ReFS by GUI and command prompt shown with examples in this guide.