CRITICAL

How to Fix Linux EIO

Quick Fix Summary

TL;DR

Check disk health with smartctl, verify cable connections, and run filesystem checks.

EIO (Input/Output Error) indicates the kernel failed to read from or write to a device, typically storage. This is a hardware-level failure signal that requires immediate investigation.

Diagnosis & Causes

  • Failing or disconnected storage hardware (HDD/SSD).
  • Corrupted filesystem or partition table.
  • Faulty or loose data/power cables.
  • RAID controller or disk array failure.
  • Underlying network storage (NFS/iSCSI) timeout or failure.
  • Recovery Steps

    1

    Step 1: Immediate Diagnostics & Log Inspection

    First, gather all available error data from the kernel and system logs to pinpoint the failing device.

    bash
    dmesg -T | grep -i EIO 
     journalctl --since "5 minutes ago" | grep -i EIO 
     lsblk 
     cat /proc/mounts | grep <suspected_device>
    2

    Step 2: Hardware Health Check & Cable Verification

    Physically inspect connections and use SMART tools to assess the disk's health. For NVMe, use nvme-cli.

    bash
    smartctl -a /dev/sdX 
     (For NVMe: nvme smart-log /dev/nvme0n1) 
     Check /sys/block/sdX/device/state
    3

    Step 3: Filesystem Check & Remount (If Read-Only)

    If the system has remounted a filesystem as read-only (ro), attempt a safe check and remount. Unmount first if possible.

    bash
    umount /failing/mountpoint 
     fsck -y /dev/sdXY 
     mount -o remount,rw /failing/mountpoint
    4

    Step 4: Failover & Data Recovery Strategy

    If the device is failing, initiate failover to a standby system or disk. Use dd_rescue to attempt data salvage from the failing block device.

    bash
    ddrescue -d -r3 /dev/failing_sdX /dev/healthy_sdY rescue.log 
     (For LVM: pvmove /dev/failing_sdX)

    Architect's Pro Tip

    "EIO on cloud/VM storage often indicates underlying hypervisor or network issues. Check your provider's status and consider detaching/reattaching the volume."

    Frequently Asked Questions

    Can a Linux EIO error cause data loss?

    Yes. EIO signals a failed physical write or read. Any data not yet committed to stable storage is at risk. Immediate read-only mode and backup are critical.

    How do I distinguish a hardware EIO from a software one?

    Check kernel logs (dmesg). Consistent errors on a specific block device (e.g., /dev/sdb) point to hardware. Isolated errors on a single file may indicate localized corruption.

    The system is frozen with EIO errors. What's the safest next step?

    If possible, trigger a kernel panic to get a crash dump for analysis: `echo c > /proc/sysrq-trigger`. Otherwise, a hard reboot may be necessary to regain control.

    Related Linux Guides