Data deduplication scrubbing job stuck on 30% for 6 days

Hi everyone
I have a LUN which has been stuck on "scrubbing" for around 6 days. It won't progress any further. I've run a scan disk and it doesn't report any errors. The second LUN on the box is running fine as well.
The logs under application and service logs -> microsoft -> windows -> deduplication doesn't report any errors.
Any ideas? Is it safe to reboot the machine while it is scrubbing?

Hi,
Based on your description, we can try to use the Get-DedupStatus and
Get-DedupVolume Windows PowerShell cmdlets to check the status of the deduplication process.
Regarding this point, the following article can be referred to for more information.
Monitor and Report for Data Deduplication
http://technet.microsoft.com/en-us/library/hh831505.aspx
Best regards,
Frank Shen

Similar Messages

W2K12-R deduplication optimization job stuck at 100% for days

Hello,
I've W2K12-R server (VM, 4 vCPU, 14 GB vRAM) with a 7 TB D: drive, deduplication is enabled for this drive. I use this drive for backing up virtual machines, at the moment 3,6 TB are used. Each night ~2 TB new data is backed up, only the C: drives of VMs,
so there is a very high dedup rate, these new 2 TB are usually deduplicated during daytimes.
Capacity                 : 6.64 TB
FreeSpace               : 3.01 TB
UsedSpace              : 3.63 TB
UnoptimizedSize        : 110.74 TB
SavedSpace             : 107.1 TB
SavingsRate           : 96 %
Since a few weeks I've the problem that ThroughputOptimization jobs (all Optimization jobs) do not finish. The job is able to free space but it stays at 100% for days.
Although I've set the DurationHours to 6 or 9 hours, the scheduled job does not end and thus no new Optimization job starts. That leads to a completely filled D drive because the space that gets filled by backup jobs during the night is not freed during
the planned time frame.
Event Log from the start of the job:
+
System
Provider
[ Name]
Microsoft-Windows-Deduplication
[ Guid]
{F9FE3908-44B8-48D9-9A32-5A763FF5ED79}
EventID
6148
Version
0
Level
4
Task
0
Opcode
0
Keywords
0x8000000000000000
TimeCreated
[ SystemTime]
2014-07-21T14:20:04.531120000Z
EventRecordID
2768
Correlation
Execution
[ ProcessID]
4672
[ ThreadID]
268
Channel
Microsoft-Windows-Deduplication/Operational
Computer
xxxxxx.net
Security
[ UserID]
S-1-5-18
EventData
JobType
1
JobInstanceId
{39D260C4-32D0-4511-8982-598DA53DB423}
VolumeGuidPath
\\?\Volume{27fe6999-df3b-4864-81fe-853126e2c9cc}\
VolumeDisplayName
D:
AvailableMemoryMb
11058
JobPriorityType
3
I don't see further information about job 39D260C4-32D0-4511-8982-598DA53DB423 in event log. In resource monitor I see that fsdmhost has very little disk activity (600 Bytes/s), but a cpu load of 25% (which is one of the four vCPUs).
I stopped the backups this week and started a manual full Optimization job, which stays at 100% since more than one day now.
Any ideas what to check? Any patch I can apply?
Some details:
23.07.014 - 8:40 Uhr
Get-DedupStatus
FreeSpace    SavedSpace   OptimizedFiles     InPolicyFiles      Volume
3.01 TB      107.1 TB     12232              12232              D:
Get-DedupStatus | fl
Volume                             : D:
VolumeId                           : \\?\Volume{27fe6999-df3b-4864-81fe-853126e2c9cc}\
Capacity                           : 6.64 TB
FreeSpace                          : 3.01 TB
UsedSpace                          : 3.63 TB
UnoptimizedSize                    : 110.74 TB
SavedSpace                         : 107.1 TB
SavingsRate                        : 96 %
OptimizedFilesCount                : 12232
OptimizedFilesSize                 : 110.58 TB
OptimizedFilesSavingsRate          : 96 %
InPolicyFilesCount                 : 12232
InPolicyFilesSize                  : 110.58 TB
LastOptimizationTime               : 23.07.2014 04:44:59
LastOptimizationResult             : 0x8056533D
LastOptimizationResultMessage      : The operation was cancelled.
LastGarbageCollectionTime          : 20.07.2014 05:26:12
LastGarbageCollectionResult        : 0x00000000
LastGarbageCollectionResultMessage : The operation completed successfully.
LastScrubbingTime                  : 20.07.2014 16:06:29
LastScrubbingResult                : 0x00000000
LastScrubbingResultMessage         : The operation completed successfully.
Get-DedupVolume
Enabled            UsageType          SavedSpace           SavingsRate
Volume
True               Default            107.1 TB             96 %
D:
Get-DedupVolume |fl
Volume                   : D:
VolumeId                 : \\?\Volume{27fe6999-df3b-4864-81fe-853126e2c9cc}\
Enabled                  : True
UsageType                : Default
DataAccessEnabled        : True
Capacity                 : 6.64 TB
FreeSpace                : 3.01 TB
UsedSpace                : 3.63 TB
UnoptimizedSize          : 110.74 TB
SavedSpace               : 107.1 TB
SavingsRate              : 96 %
MinimumFileAgeDays       : 0
MinimumFileSize          : 32768
NoCompress               : False
ExcludeFolder            :
ExcludeFileType          :
ExcludeFileTypeDefault   : {edb, jrs}
NoCompressionFileType    : {asf, mov, wma, wmv...}
ChunkRedundancyThreshold : 100
Verify                   : False
OptimizeInUseFiles       : False
OptimizePartialFiles     : False
Get-DedupMetadata | fl
Volume                         : D:
VolumeId                       : \\?\Volume{27fe6999-df3b-4864-81fe-853126e2c9cc}\
StoreId                        : {1E87DD60-0265-4726-9EDD-81FCE4E9D70A}
DataChunkCount                 : 151954987
DataContainerCount             : 3983
DataChunkAverageSize           : 24.35 KB
DataChunkMedianSize            : 0 B
DataStoreUncompactedFreespace : 0 B
StreamMapChunkCount            : 95398
StreamMapContainerCount        : 1944
StreamMapAverageDataChunkCount :
StreamMapMedianDataChunkCount :
StreamMapMaxDataChunkCount     :
HotspotChunkCount              : 1048520
HotspotContainerCount          : 54
HotspotMedianReferenceCount    :
CorruptionLogEntryCount        : 0
TotalChunkStoreSize            : 3.6 TB
Get-DedupSchedule
Enabled    Type               StartTime              Days
Name
True       Optimization
BackgroundOptimization
False      GarbageCollection 07:30                  {Monday, Tuesda... DailyGarbageCollection
True       Optimization       09:00                  {Sunday, Monday... ThroughputOptimization
False      Optimization       14:30                  {Sunday, Monday... ThroughputOptimization-2
True       GarbageCollection 16:00                  Saturday           WeeklyGarbageCollection
True       Scrubbing          03:45                  Sunday
WeeklyScrubbing
Get-DedupSchedule | fl
Name                     : BackgroundOptimization
Enabled                  : True
Type                     : Optimization
Days                     :
Start                    :
DurationHours            :
StopWhenSystemBusy       : True
Memory                   : 25 %
Priority                 : Low
InputOutputThrottleLevel : Low
ScheduledTask            : \Microsoft\Windows\Deduplication\BackgroundOptimization
Full                     :
ReadOnly                 :
Name                     : DailyGarbageCollection
Enabled                  : False
Type                     : GarbageCollection
Days                     : {Monday, Tuesday, Wednesday, Thursday...}
Start                    : 17.06.2014 07:30:00
DurationHours            : 0
StopWhenSystemBusy       : False
Memory                   : 50 %
Priority                 : Normal
InputOutputThrottleLevel : None
ScheduledTask            : \Microsoft\Windows\Deduplication\DailyGarbageCollection
Full                     : False
ReadOnly                 : False
Name                     : ThroughputOptimization
Enabled                  : True
Type                     : Optimization
Days                     : {Sunday, Monday, Tuesday, Wednesday...}
Start                    : 14.07.2014 09:00:00
DurationHours            : 6
StopWhenSystemBusy       : False
Memory                   : 40 %
Priority                 : Normal
InputOutputThrottleLevel : None
ScheduledTask            : \Microsoft\Windows\Deduplication\ThroughputOptimization
Full                     : False
ReadOnly                 : False
Name                     : ThroughputOptimization-2
Enabled                  : False
Type                     : Optimization
Days                     : {Sunday, Monday, Tuesday, Wednesday...}
Start                    : 14.07.2014 14:30:00
DurationHours            : 9
StopWhenSystemBusy       : False
Memory                   : 40 %
Priority                 : Normal
InputOutputThrottleLevel : None
ScheduledTask            : \Microsoft\Windows\Deduplication\ThroughputOptimization-2
Full                     : False
ReadOnly                 : False
Name                     : WeeklyGarbageCollection
Enabled                  : True
Type                     : GarbageCollection
Days                     : Saturday
Start                    : 20.06.2014 16:00:00
DurationHours            : 26
StopWhenSystemBusy       : True
Memory                   : 50 %
Priority                 : High
InputOutputThrottleLevel : None
ScheduledTask            : \Microsoft\Windows\Deduplication\WeeklyGarbageCollection
Full                     : True
ReadOnly                 : False
Name                     : WeeklyScrubbing
Enabled                  : True
Type                     : Scrubbing
Days                     : Sunday
Start                    : 17.05.2014 03:45:00
DurationHours            : 0
StopWhenSystemBusy       : True
Memory                   : 50 %
Priority                 : Normal
InputOutputThrottleLevel : None
ScheduledTask            : \Microsoft\Windows\Deduplication\WeeklyScrubbing
Full                     : True
ReadOnly                 : False
Get-DedupJob
Type               ScheduleType       StartTime              Progress   State
Volume
Optimization       Scheduled                                 0 %
Queued                 D:
Optimization       Manual             16:20                  100 %
Running                D:
Get-DedupJob |fl
Volume                   : D:
VolumeId                 : \\?\Volume{27fe6999-df3b-4864-81fe-853126e2c9cc}\
Type                     : Optimization
ScheduleType             : Scheduled
StartTime                :
Progress                 : 0 %
State                    : Queued
Id                       : {556DC9B2-068C-4CAE-9BE7-4104A0963F49}
StopWhenSystemBusy       : True
Memory                   : 25 %
Priority                 : Low
InputOutputThrottleLevel : Low
ProcessId                : 0
Full                     : False
ReadOnly                 : False
Volume                   : D:
VolumeId                 : \\?\Volume{27fe6999-df3b-4864-81fe-853126e2c9cc}\
Type                     : Optimization
ScheduleType             : Manual
StartTime                : 21.07.2014 16:20:04
Progress                 : 100 %
State                    : Running
Id                       : {39D260C4-32D0-4511-8982-598DA53DB423}
StopWhenSystemBusy       : False
Memory                   : 60 %
Priority                 : High
InputOutputThrottleLevel : None
ProcessId                : 4672
Full                     : True
ReadOnly                 : False

I stopped the manually started optimization job now and see these event log entries.
Data Deduplication job type "Optimization" on volume "D:" was cancelled manually.
Data Deduplication job of type "Optimization" on volume "D:" has completed with return code: 0x8056533d, The operation was cancelled.
Optimization reconciliation has completed.
Volume: D: (\\?\Volume{27fe6999-df3b-4864-81fe-853126e2c9cc}\)
Reconciled containers: 1908
Unreconciled containers: 36
Merged containers: 0
Total reconciled references: 0
Error code: 0x0
Error message: NULL
Optimization job has completed.
Volume: D: (\\?\Volume{27fe6999-df3b-4864-81fe-853126e2c9cc}\)
Error code: 0x0
Error message:
Savings rate: 96
Saved space: 117761059523158
Volume used space: 3988537147392
Volume free space: 3312769826816
Optimized file count: 12232
In-policy file count: 12232
Job processed space (bytes): 1815468262325
Job elapsed time (seconds): 150522
Job throughput (MB/second): 11.5

Installing PS in CC and has bean stuck on finishing for days

installing PS in CC and has bean stuck on finishing for days
And have tried to instal other programs mainly lightroom 6 and always get waiting text
click the more imfo botton and get taken to error page
saying sorry, this page is not available

Force quitting the App Store during a software update download deletes any partial downloads already initiated. There is no way to pause a software update in Mountain Lion, which is a huge, glaring problem for anyone with a MacBook/MacBook Pro. If I'm in the middle of a large software update download, I can either force quit the App Store and lose any partial download or wait until the download is finished, which could be a long time if its a multi-gigabyte update. The whole point of having a laptop is the ability to take it with you and anything that prevents that ability must be changed.

My px4-300r stuck at 95% for days

I have px4-300r storage device. it stuck at 95% for days. I tried the possible solutions mentioned in the similar topics (like remove disks and re-seat it , ... ) please help me even if I'll lose my data

Hello Mulham,
I would try booting the NAS with no drives installed.
If it boots, try adding the drives, one-by-one, giving about a miniute between each one, while the NAS is still powered up.
If the NAS will not boot with the drives removed, you will need to start a support ticket with a local agent:
https://lenovo-eu-en.custhelp.com/app/answers/detail/a_id/14722

Default Data Deduplication Optimisation job with "StopWhenSystemBusy" does not work correctly

I have configured Data Deduplication on Windows Server 2012 R2 and enabled on a volume.
Data Deduplication creates a default Optimisation task called "BackgroundOptimization".
I have placed files on my volume, and started the job manually. It succeeds but does nothing. InPolicyFiles still shows zero.
If I edit this job in Task Scheduler and remove the /backoff switch (StopWhenSystemBusy), then start the job manually again, it succeeds, and also incrememnts the InPolicyFiles.
At the same time, the CPU and RAM on the server is fairly idle, with no users on the system.
Is there a problem with the StopWhenSystemBusy? Should we not use this parameter, and simply run the task out of business hours instead? Has anyone seen this before?
This happens on two Windows Server 2012 R2 servers, so is reproducable here.
David

Hi David,
Please also try -Priority and -Memory with-StopWhenSystemBusy - set Priority to High and set a proper memory value such as 50% to see the result.
If you have any feedback on our support, please send to [email protected]

How can i delete old print jobs stuck in queue for printer no longer present?

I've deleted the old hp printer from control panel (printers and devices) but the old stuck print jobs are still hanging out there and keep me from printing anything on the new printer. Help please. New printer is Dell V715w

Hi there karen-c,
Could you provide the community with a little more information to help narrow troubleshooting? What operating system is this?
You can say thanks by clicking the Kudos Star in my post. If my post resolves your problem, please mark it as Accepted Solution so others can benefit too.

Since update to 8.2, iphone 6 stuck sending message for days

SSince I did the update the other day, randomly when sending text messages, the message will read "sending", but never complete or go through. I've had several that have been sending since Friday (it's Sunday now). I have reset the network settings, nothing. I have even erased all my messages, nothing.
Again, it's random, so sometimes it goes with one name and then I delete and restart the phone and it goes through. Other times it's different contacts. can be text or photos, other times they go through. I'm on AT&T, and it does not matter the carrier it goes to either.
WHen hen people send me text, they say it shows that I have received them, but I have not. That's not good with the wife as an excuse!
What can can I do?

I have the exact same issue! I kept turning my phone on and off and every time I restarted it, it would work for a while and then stop. It's been sending my messages well for the last day but all my apps are crashing. I can't use any of them.

I've been stuck on this for days

i have attached a link to some files in hopes of getting
help.
http://www.garyhush.com/post/projectFolder.zip
trouble.fla is the main document and sOne.fla is a swf that
loads into it.
the pink button in sOne.swf gives me this error
Error #1009: Cannot access a property or method of a null
object reference.
at sOne_fla::MainTimeline/onClickz()
my goal is to move all the thumbnail buttons into a swf
similiar to sOne, and therefore be able to show more images,
different groups of images, by selecting groups of thumbnails to
load onto the stage.
otherwise is is basically ready to get the job done.

i have attached a link to some files in hopes of getting
help.
http://www.garyhush.com/post/projectFolder.zip
trouble.fla is the main document and sOne.fla is a swf that
loads into it.
the pink button in sOne.swf gives me this error
Error #1009: Cannot access a property or method of a null
object reference.
at sOne_fla::MainTimeline/onClickz()
my goal is to move all the thumbnail buttons into a swf
similiar to sOne, and therefore be able to show more images,
different groups of images, by selecting groups of thumbnails to
load onto the stage.
otherwise is is basically ready to get the job done.

Data Deduplication - max memory allocation

Hi
i encountered this issue on a server:
2012R2
https://support.microsoft.com/en-us/kb/2891882/
i have configured the value from the KB, but i still get:
Data Deduplication detected job type "Optimization" on volume "\\?\Volume{8ce5ac3c-46df-4a37-8a9e-c4b28f1354d1}\" uses too much memory. 3179 MB is assigned. 3870 MB is used.
Data Deduplication cancelled job type "Optimization" on volume "\\?\Volume{8ce5ac3c-46df-4a37-8a9e-c4b28f1354d1}\". It uses too much memory than the amount assigned to it.
for the volume with most data.
can the value be increased?

Hi,
Please try to attempt a workaround by adding the following registry key with value 1000 to resolve the issue.
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ddpsvc\Settings\WlmMemoryOverPercentThreshold
Best Regards,
Mandy
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact [email protected]

GRC AC 5.3 - Sync Mode background job running for days.

We have the following performance issues on the GRC System.
1- Please note that the sync mode background job has been running for DAYS.
My question is if I truncate table virsa_cc_prmvl; (which stores all the analysis results), will my GRC system be effected.
2- Secondly we have a job analysing roles, which causes the performance to go down. And due to we have a extremely high number of roles, we would like to know if there would be a way to improve the analysis for roles. Is there any parameter modification in GRC / J2EE/ backend related to roles?
Details of our roles:
- we have more than 60000 roles in our system
- 75 % of the roles are not assigned to any user (is there a way to avoid these roles? these roles are not put in order so too tedious to choose roles individually or by range.
3- We are running a job with four risks and it's taking more than two minutes per user (we've got 10.000 + users in total)
How can I resolve this performance issue.
4- What is the recommended size of the batch for user sincronization. Should I use 1000 or 10,000 (based on our above environment).
Thanks,
Haleem

Hello, Abdul!
How you have solved a problem?
Thank you a lot.
Best regards,
Denis Gaykov

Data Deduplication, Defrag & DPM 2012 R2 Churn

We have a protected 2012 R2 server that has Data Deduplication enabled. DPM tried to create a recovery point and ran out of disk space. When I check to see what happened, the job had transferred over 43GB of data. This is way outside of the norm
for this protection group. Upon investigation I found that a scheduled weekly defrag had run. I assume this to be the source of the churn. Can anyone provide an guidance on the interaction of the these processes? The scheduled defrag
was enabled by default. I assume we'll need to disable this? I have never defragged my VM's as it would cause massive churn for the VM backup's in DPM. But this was not a VM backup, just a backup of files on a deduplicated volume.
Thanks,
Scott

Thanks for the reply. It's pretty simple really. I have a 2012 R2 file server with data deduplication enabled on one of the volumes. There was a scheduled defrag that ran on the volume and appears to have caused massive churn for the
DPM recovery point. Question is if this is unique to data deduplication and DPM? The same problem occurs when you defrag a VM that's being backed up by DPM. That's why we don't defrag our VHDs. Normal file servers without data deduplication
do not exhibit this behavior. The scheduled defrag was set by default and I'm assuming needs to be disabled. I'm just looking for guidance on why this is an issue.

Deduplication error: there was a problem retrieving data deduplication schedule

Hi,
After changing dedup schedule once we are receiving this error and not able to change it any more.
there was a problem retrieving data deduplication schedule

Hi,
After changing dedup schedule once we are receiving this error and not able to change it any more.
there was a problem retrieving data deduplication schedule
Take a look @ PowerShell cmdlets for dedupe here:
PowerShell dedupe cmdlets
http://technet.microsoft.com/en-us/library/hh848450.aspx
Of your interest are Remove-DedupeSchedule and New-DedupSchedule ones. Try removing set schedule
you have now, making sure the status is "no jobs" and then create new schedule to see would it make the difference.
Good luck!
StarWind VSAN [Virtual SAN] clusters Hyper-V without SAS, Fibre Channel, SMB 3.0 or iSCSI, uses Ethernet to mirror internally mounted SATA disks between hosts.

Data Deduplication

Hi everyone,
I am having trouble to experience data deduplication in saving storage space.
I basically created a virtual server, Windows Server 2012 R2 and only has Data deduplication enabled on the server roles/features. An extra virtual SCSI disk 5GB and set it as K: volume, two .exe files in the volume taking rougly 1GB of space.
When I run the data deduplication task and the job processed in less than 10 seconds.
What have I done wrong?
Thanks guys

Hi,
How do you set the schedule? By default it will not optimize data created less than x days (7 days if I'm correct).
Also generally EXE files will not be dedupped - whether they are same exe file? If not, it is common that no file get dedupped as EXE file cannot be separated like TXT or DOC files.
Please remember to mark the replies as answers if they help and un-mark them if they provide no help. If you have feedback for TechNet Support, contact [email protected]

User Profile Service - User Profile Incremental Synchronization Timer job stuck at 33% Status: Pausing

User Profile Service - User Profile Incremental Synchronization Progress: 33% Status: Pausing
It has been almost 15 days.
Both User Profile Service and User Profile Synchronization Service are in Started state and FIM service also starting
I tried clearing sharePoint config cache.
I also restarted the sharepoint timer service.
I tried almost everything that is on Internet but nothing helped me.
Is there any other way to solve the issue as I was struck on production server (ASAP)
In synchronization serivce manager status of MOSS_DeltaImport is Inprogress from past 2 days
Best Regards.

Hi,
Please follow the steps in the link below to clear the configuration cache.
http://blogs.msdn.com/b/jamesway/archive/2011/05/23/sharepoint-2010-clearing-the-configuration-cache.aspx
Here is a similar thread for your reference:
https://social.technet.microsoft.com/Forums/en-US/beaa852c-6f40-428a-b97c-20722864e045/user-profile-service-user-profile-incremental-synchronization-timer-job-stuck-at-88-status?forum=sharepointadminprevious
Or try to clear the file system cache on all servers in the server farm on which the Windows SharePoint Services Timer service is running. Microsoft has provided a step by step procedure on clearing file system cache from the SharePoint front-end servers
in this kb article.
You can also see the ULS logs and check error messages.
http://sharepointlogviewer.codeplex.com/
Best Regards
Dennis Guo
TechNet Community Support

Short dump while executing job SM:EXEC SERVICES for EWA alert generation

Hello All,
I'm getting short dump error while executing job SM:EXEC SERVICES for EWA alert generation.I'm using EHP1@SP22 of Sol Man.
Please find the logs
Error in the ABAP Application Program
The current ABAP program "SAPLDSVAS_PROC" had to be terminated because it has
come across a statement that unfortunately cannot be executed.
The following syntax error occurred in program "RDSVASCABAP_TRANS__________073
" in include "RDSVASIABAP_TRANS__________073 " in
line 1782:
"Field "STATEMENT_SUMMARY001019" is unknown. It is neither in one of th"
"e specified tables nor defined by a "DATA" statement. "DATA" statement"
The include has been created and last changed by:
Created by: "SAP "
Last changed by: "SAP "
Error in the ABAP Application Program
The current ABAP program "SAPLDSVAS_PROC" had to be terminated because it has
come across a statement that unfortunately cannot be executed.
If anyone came across such kind of dump, please reply asap, due to this, i could not generate EWA report for R/3 system 6.04.
Thanks,
Anand.

>
Anand Tigadikar wrote:
> Thx Paul, but i already did checked this note and its not solving my issue
Dear Anand,
Did you perform a complete service definition replacement for SDCC/SDCCN as described in the note?
These are manual steps you need to perform on the system and are not code corrections.
I would suggest you perform it, if you have not, and check the SDCCN logs to review the process ran cleanly.
Note: Please make sure SDCC_OSS is functioning and a connection to SAP can be made before
deleting the service definitions.
If you have already replaced the service defintions on the Solution Manager system, then no point doing it again. However checking a note and applying it are not the same thing. So I am uncertain if you replaced the service definitions or not. The dump you are getting in SM:EXEC SERVICES suggests its due to a problem with the service definitons. The recommendation is to replace them according to the process described in SAP Note 727998. If you have not done so please do so.
If you have successfully replaced the service definitions are you still getting the same dump, or has it possibly changed?
Regards,
Paul

Data deduplication scrubbing job stuck on 30% for 6 days

Similar Messages

Maybe you are looking for