Error: NDMP operation failed due to media error

Question

I scheduled a RMAN backup job on oracle 11g EM and sometime it failed when running datafile backup.  The log showed the error come from media layer and did not specify detail message. We used secure backup to transfer data to tape. There are 4 tape drivers in service, but only one of them always return this error.  Could anyone tell how to resolve the problem ?
RMAN-03009: failure of backup command on c1 channel at 01/23/2014 11:17:29
ORA-27192: skgfcls: sbtclose2 returned error - failed to close file
ORA-19511: Error received from media manager layer, error text:
   sbtclose2: Internal error - NDMP Data Service has exited prematurely.
   For more information, please check the transcript for this job ('oracle/33779.1').
ORA-27000: skgfqsbi: failed to initialize storage subsystem (SBT) layer
Additional information: 1292
ORA-19511: Error received from media manager layer, error text:
continuing other job steps, job failed will not be re-run
And I found below logs in the secure backup scripts log. it that mean a network error?
10:16:42 await_ndmp_event: sending progress update
    10:16:42 SPU: sending progress update
    10:17:43 QTOS: received osb_stats message for job oracle/33656.1, kbytes 10452672, nfiles 0
10:17:43 await_ndmp_event: sending progress update
    10:17:43 SPU: sending progress update
    10:19:35 NWEM: saw tape write error log message ("tape write failed due to media error")
tape write failed due to media error
    10:19:35 MNPO: mover halted with reason=media error
    10:19:35 MGS:  ms.record_size 65536, ms.record_num 0x28094, ms.bytes_moved 0x280940000
    10:19:35 QTOS: received osb_stats message for job oracle/33656.1, kbytes 10497280, nfiles 0
    10:19:35 MNPO: data service halted with reason=connection error
    10:19:35 SNPD: Data Service reported bytes processed 0x280B40000
    10:19:53 A_T:  suppressing filemark output due to NDMP having written one
    10:19:53 A_T:  writing marker label; here it is:

Answer 1

Judging by the transcript snips that you provided, I also believe that this is caused by a bad drive or bad media.
Start by looking at the output of the command :
  # obtool dumpdev <tape_drive_name>
(replace "<tape_drive_name>" by the name of the drive which was used during the failed job).
Look for any errors or warnings during the date/time the backup job failed. These message usually provide a good idea about the problem that occurred.
Next try the same backup using the same media in another drive to see it the problem reoccurs or not. In case the problem reoccurs, there's a good chance that the media is bad. If the backup succeeds on another drive, then try the same backup using the original drive but with a new media. If the backup fails again on the same drive using new media, you should open a support request with the tape drive vendor. In that case, collect the logs from the tape library (and dump the tape drives is possible).
HTH,
Gio