X230: SATA errors with 180GB Intel 520 SSD under heavy write load

Question

Hi,
We spent a couple of days cornering what appears to be an issue with the Intel 520 SSD drives in Lenovo x230 laptops. It was first showing up on a clean Debian installation, while installing a guest operating system into a VM. Looking around on forums, there appears to be some people having issues with database workloads too. So I decided to create a small user-space program to repoduce the problem. IMPORTANT: Before you try it, be ready for a system crash requiring a cold reboot.. It's available at:
git://git.efficios.com/test-ssd.git
direct link to .c file:
https://git.efficios.com/?p=test-ssd.git;a=blob;f=test-ssd-write.c;hb=refs/heads/master
This program simply performs random-access-writes of 4Kb into a single file.
Executive summary of our findings (the details are in the test-ssd-write.c header in the git repo):
- We reproduced this issue on 4 x230 machines (all our x230 have 180GB Intel drives, and they are all affected),
- We took a SSD from one of the machines, moved it into an x200, and the problem still occurs,
- The problem seems to occur independently of the filesystem (reproduced on ext3 and ext4),
- Problem reproduced by test-ssd-write.c (git tree above): After less than 5 minutes of the heavy write workload, we get SATA errors and we need to cold reboot the machine to access the drive again. Example usage (don't forget to prepare for a computer freeze):
  ./test-ssd-write somefileondisk 209715200 1234 -z
  (see options by just running ./test-ssd-write)
The problem occurs with drive model SSDSC2BW180A3L, with both firmwares LE1i and LF1i (those are Lenovo firmwares). We could reproduce the issue on 3.2 (Debian), 3.5 (Debian), 3.7.9 (Arch) distribution kernels. We could reproduce it with x230 BIOS G2ET90WW (2.50) 2012-20-12 and G2ET86WW (2.06) 2012-11-13, but since it can be reproduced on a x200 too, it does not appear to be a BIOS issue.
We tried the program on a range of other SSD drives, one of those including the same SandForce 2281 controller (details within test-ssd-write.c header). So our current guess is that the Lenovo firmware on the SSD might be part of the problem, but it might be good if we could to confirm that Intel's firmwares work fine.
Thoughts, ideas, hints about who to contact on this issue would be very much welcome,
Thanks,
Mathieu

Answer 1

I can confirm the exact same problem with Debian and Ubuntu.
I can reproduce it easily on Debian by installing a VM.
On Ubuntu it is harder to reproduce but I already hit this problem when installing updates (dpkg does a lot of sync/fdatasync just like your test program).
I think the difference between the 2 distros that explain why it is easier to reproduce on one is the default I/O scheduler (cfq for Debian and deadline for Ubuntu).
Lenovo already completely replaced my laptop (X230 with SSDSC2BW180A3L) and I still have this problem.
We need a firmware fix.
Thanks !