Invalid Argument When Trying to Write Direct I/O to a 4K Native Sector Size Device

Direct I/O Invalid Argument

When trying to write Direct I/O to a particular block device, I ran into this ugly error:

# dd if=/dev/zero of=/dev/sdb oflag=direct
dd: writing to ‘/dev/sdb’: Invalid argument
1+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000202393 s, 0.0 kB/s

After some investigation, it turns out the cause was the device was actually configured and low level formatted to the newer 4K Native Sector Size (aka 4Kn Advanced Format). Older devices are all 512 byte sector size.

The problem is that the default block size for many applications is still 512 bytes. The solution is easy, once you know the cause. You just simply write blocks in 4K byte size.

# dd if=/dev/zero of=/dev/sdb oflag=direct bs=4k
^C
73063+0 records in
73063+0 records out
299266048 bytes (299 MB) copied, 4.83917 s, 61.8 MB/s

Is My Device 4K Sector Size?

Many newer storage devices may be configured for 512, 512e or 4K.  So how can you tell if you have a disk device that is in 4K native sector size? There are several options:

1) fdisk will tell you:

# fdisk /dev/sdb
Note: sector size is 4096 (not 512)
...

2) blockdev can tell you:

# blockdev --help
 --getss                   get logical block (sector) size
 --getpbsz                 get physical block (sector) size

# ## 512 mode:
# blockdev --getss /dev/sda
512
# blockdev --getpbsz /dev/sda
512

# ## 4Kn mode:
# blockdev --getss /dev/sdb
4096
# blockdev --getpbsz /dev/sdb
4096

# ## 512e mode:
# blockdev --getss /dev/sdc
512
# blockdev --getpbsz /dev/sdc
4096

3) Your storage vendor's tools should tell you.  Many times the storage vendor will also provide tools to low level format to either 512, 512e or 4K

Why 4Kn?

So why are some storage vendors providing newer storage devices in 4K sector size?  Efficiency, and Data Integrity.

"Larger sectors use the storage surface area more efficiently for large files but less efficiently for smaller files, and enable the integration of stronger error correction algorithms to maintain data integrity at higher storage densities." (source)

Summary:

• Greater storage efficiency for larger files (but less efficient for smaller files)
• Better bandwidth performance (large block read/writes), but IOPS (small block read/writes) will suffer
• Improved error correction algorithms to maintain data integrity at higher storage densities

512e Alternative?

For Operating Systems and applications that may not support 4K native sector size, vendors may also provide a 512e mode (512 emulation). 512e mode is a 512 byte local sector size front-end emulation with 4K byte physical sector size back-end.

The Operating System would hand down a 512 byte request as usual, but then the storage controller would convert the request to a 4K byte request. This provides the efficiency and data integrity benefits for the physical storage, but may come with a performance hit, due to the emulation and conversion required (especially if not aligned properly).

This mode also allows a stepping stone for storage vendors to move forward with 4K technology, while still supporting older Operating Systems and applications.