Dense Restructure 1070020 Out of disk space. Can't create new data file

Question

During a Dense Restructure we receive: Error(1070020) Out of disk space. Cannot create a new [Data] file.
Essbase 6.5.3 32-bit
Windows 2003 32bit w/16GB RAM
Database is on E: drive with 660GB space total, database is ~220GB.
All cubes are unlimited
Tried restoring from backup same problem.
Over years and years the database is never recalculated, never exported and imported, never verified. Only new data loaded and dense restructured.
Towards the end of a dense restructure (about 89 pan files through about 101 2GB pag files), getting an error: Error(1070020) Out of disk space. Cannot create a new [Data] file.
There are still several hundred GB of free space available, and we can write to this free space outside of the essbase application within windows.
The server's file system is consistent, defragmented, and can prove use of additional space. Hard drive controller and system does not report any "hardware issues".
Essbase.cfg file
; The following entry specifies the full path to JVM.DLL.
JvmModuleLocation C:\Hyperion\Essbase\java\jre13\bin\hotspot\jvm.dll
;This statement loads the essldap.dll as a valid authentication module
;AuthenticationModule LDAP essldap.dll x
DATAERRORLIMIT 30000
;These settings are here to deal with error 1040004
NETRETRYCOUNT 2000
NETDELAY 1600
App log
[Sat Oct 17 13:59:32 2009]Local/removedfrompost/removedfrompost/admin/Info(1007044)
Restructuring Database [removedfrompost]
[Sat Oct 17 15:48:42 2009]Local/removedfrompost/removedfrompost/admin/Error(1070020)
Out of disk space. Cannot create a new [Data] file. [adIndNewFile] aborted
[Sat Oct 17 15:48:42 2009]Local/removedfrompost///Info(1008108)
Essbase Internal Logic Error [7333]
[Sat Oct 17 15:48:42 2009]Local/removedfrompost///Info(1008106)
Exception error log [C:\HYPERION\ESSBASE\app\removedfrompost\log00002.xcp] is being created...
log00002.xcp
Assertion Failure - id=7333 condition='((!( dbp )->bFatalError))'
- line 11260 in file datbuffm.c
- arguments [0] [0] [0] [0]
Additional log info from database start to restructure failure
Starting Essbase Server - Application [removedfrompost]
Loaded and initialized JVM module
Reading Application Definition For [removedfrompost]
Reading Database Definition For [removedfrompost]
Reading Database Definition For [TempOO]
Reading Database Definition For [WTD]
Reading Database Mapping For [removedfrompost]
Writing Application Definition For [removedfrompost]
Writing Database Definition For [removedfrompost]
Writing Database Definition For [TempOO]
Writing Database Definition For [WTD]
Writing Database Mapping For [removedfrompost]
Waiting for Login Requests
Received Command [Load Database]
Writing Parameters For Database [removedfrompost]
Reading Parameters For Database [removedfrompost]
Reading Outline For Database [removedfrompost]
Declared Dimension Sizes = [289 125 2 11649 168329 1294 622 985 544 210 80 2016 11 9 9 8 8 1 1 6 1 3 1 2 2 1 2 1 2 77 2 65 1 1 1 1 1 1 1 1 1 1 1 260 4 3018 52 6 39 4 1577 6 ]
Actual Dimension Sizes = [289 119 1 1293 134423 1294 622 985 544 210 80 2016 11 9 9 8 8 1 1 6 1 3 1 2 2 1 2 1 2 77 2 65 1 1 1 1 1 1 1 1 1 1 1 260 4 3018 52 6 39 4 1577 5 ]
The number of Dynamic Calc Non-Store Members = [80 37 0 257 67 ]
The number of Dynamic Calc Store Members = [0 0 0 0 0 ]
The logical block size is [34391]
Maximum Declared Blocks is [1960864521] with data block size of [72250]
Maximum Actual Possible Blocks is [173808939] with data block size of [17138]
Formula for member [4 WK Avg Total Sls U] will be executed in [CELL] mode
Formula for member [Loc Cnt] will be executed in [CELL] mode
Formula for member [OH Str Cnt] will be executed in [CELL] mode
Formula for member [Current Rtl] will be executed in [CELL] mode
Essbase needs to retrieve [1017] Essbase Kernel blocks in order to calculate the top dynamically-calculated block.
The Dyn.Calc.Cache for database [removedfrompost] can hold a maximum of [76] blocks.
The Dyn.Calc.Cache for database [removedfrompost], when full, will result in [allocation from non-Dyn.Calc.Cache memory].
Writing Parameters For Database [removedfrompost]
Reading Parameters For Database [removedfrompost]
Unable to determine the amount of virtual memory available on the system
Index cache size ==> [1048576] bytes, [128] index pages.
Index page size ==> [8192] bytes.
Using buffered I/O for the index and data files.
Using waited I/O for the index and data files.
Unable to determine the amount of virtual memory available on the system
Reading Data File Free Space Information For Database [removedfrompost]...
Data cache size ==> [3145728] bytes, [22] data pages
Data file cache size ==> [0] bytes, [0] data file pages
Missing Database Config File [C:\HYPERION\ESSBASE\APP\removedfrompost\removedfrompost\removedfrompost.cfg], Query logging disabled
Received Command [Get Database Volumes]
Received Command [Load Database]
Writing Parameters For Database [TempOO]
Reading Parameters For Database [TempOO]
Reading Outline For Database [TempOO]
Declared Dimension Sizes = [277 16 2 1023 139047 ]
Actual Dimension Sizes = [277 16 1 1022 138887 ]
The number of Dynamic Calc Non-Store Members = [68 3 0 0 0 ]
The number of Dynamic Calc Store Members = [0 0 0 0 0 ]
The logical block size is [4432]
Maximum Declared Blocks is [142245081] with data block size of [8864]
Maximum Actual Possible Blocks is [141942514] with data block size of [2717]
Essbase needs to retrieve [1] Essbase Kernel blocks in order to calculate the top dynamically-calculated block.
The Dyn.Calc.Cache for database [TempOO] can hold a maximum of [591] blocks.
The Dyn.Calc.Cache for database [TempOO], when full, will result in [allocation from non-Dyn.Calc.Cache memory].
Writing Parameters For Database [TempOO]
Reading Parameters For Database [TempOO]
Unable to determine the amount of virtual memory available on the system
Index cache size ==> [1048576] bytes, [128] index pages.
Index page size ==> [8192] bytes.
Using buffered I/O for the index and data files.
Using waited I/O for the index and data files.
Unable to determine the amount of virtual memory available on the system
Reading Data File Free Space Information For Database [TempOO]...
Data cache size ==> [3145728] bytes, [144] data pages
Data file cache size ==> [0] bytes, [0] data file pages
Missing Database Config File [C:\HYPERION\ESSBASE\APP\removedfrompost\TempOO\TempOO.cfg], Query logging disabled
Received Command [Get Database Volumes]
Received Command [Load Database]
Writing Parameters For Database [WTD]
Reading Parameters For Database [WTD]
Reading Outline For Database [WTD]
Declared Dimension Sizes = [2 105 2 11649 158778 1279 609 971 531 208 78 2017 11 9 9 1 1 1 1 6 1 2 1 1 2 1 1 1 2 77 1 1 1 1 1 1 1 1 1 1 1 1 1 260 3 2954 52 6 39 4 1581 6 ]
Actual Dimension Sizes = [1 99 1 1293 127722 1279 609 971 531 208 78 2017 11 9 9 1 1 1 1 6 1 2 1 1 2 1 1 1 2 77 1 1 1 1 1 1 1 1 1 1 1 1 1 260 3 2954 52 6 39 4 1581 5 ]
The number of Dynamic Calc Non-Store Members = [0 29 0 257 57 ]
The number of Dynamic Calc Store Members = [0 0 0 0 0 ]
The logical block size is [99]
Maximum Declared Blocks is [1849604922] with data block size of [420]
Maximum Actual Possible Blocks is [165144546] with data block size of [70]
Formula for member [Loc Cnt] will be executed in [CELL] mode
Formula for member [OH Str Cnt] will be executed in [CELL] mode
Formula for member [Current Rtl] will be executed in [CELL] mode
Essbase needs to retrieve [1017] Essbase Kernel blocks in order to calculate the top dynamically-calculated block.
The Dyn.Calc.Cache for database [WTD] can hold a maximum of [26479] blocks.
The Dyn.Calc.Cache for database [WTD], when full, will result in [allocation from non-Dyn.Calc.Cache memory].
Writing Parameters For Database [WTD]
Reading Parameters For Database [WTD]
Unable to determine the amount of virtual memory available on the system
Index cache size ==> [1048576] bytes, [128] index pages.
Index page size ==> [8192] bytes.
Using buffered I/O for the index and data files.
Using waited I/O for the index and data files.
Unable to determine the amount of virtual memory available on the system
Reading Data File Free Space Information For Database [WTD]...
Data cache size ==> [3145728] bytes, [5617] data pages
Data file cache size ==> [0] bytes, [0] data file pages
Missing Database Config File [C:\HYPERION\ESSBASE\APP\removedfrompost\WTD\WTD.cfg], Query logging disabled
Received Command [Get Database Volumes]
Received Command [Set Database State]
Writing Parameters For Database [removedfrompost]
Writing Parameters For Database [removedfrompost]
Received Command [Get Database State]
Received Command [Get Database Info]
Received Command [Set Database State]
Writing Parameters For Database [TempOO]
Writing Parameters For Database [TempOO]
Received Command [Get Database State]
Received Command [Get Database Info]
Received Command [Set Database State]
Writing Parameters For Database [WTD]
Writing Parameters For Database [WTD]
Received Command [Get Database State]
Received Command [Get Database Info]
Received Command [SetApplicationState]
Writing Application Definition For [removedfrompost]
Writing Database Definition For [removedfrompost]
Writing Database Definition For [TempOO]
Writing Database Definition For [WTD]
Writing Database Mapping For [removedfrompost]
User [admin] set active on database [removedfrompost]
Clear Active on User [admin] Instance [1]
User [admin] set active on database [removedfrompost]
Received Command [Restructure] from user [admin]
Reading Parameters For Database [Drxxxxxx]
Reading Outline For Database [Drxxxxxx]
Reading Outline Transaction For Database [Drxxxxxx]
Declared Dimension Sizes = [289 126 2 11649 168329 1294 622 985 544 210 80 2016 11 9 9 8 8 1 1 6 1 3 1 2 2 1 2 1 2 77 2 65 1 1 1 1 1 1 1 1 1 1 1 260 4 3018 52 6 39 4 1577 6 ]
Actual Dimension Sizes = [289 120 1 1293 134423 1294 622 985 544 210 80 2016 11 9 9 8 8 1 1 6 1 3 1 2 2 1 2 1 2 77 2 65 1 1 1 1 1 1 1 1 1 1 1 260 4 3018 52 6 39 4 1577 5 ]
The number of Dynamic Calc Non-Store Members = [80 37 0 257 67 ]
The number of Dynamic Calc Store Members = [0 0 0 0 0 ]
The logical block size is [34680]
Maximum Declared Blocks is [1960864521] with data block size of [72828]
Maximum Actual Possible Blocks is [173808939] with data block size of [17347]
Formula for member [4 WK Avg Total Sls U] will be executed in [CELL] mode
Formula for member [Loc Cnt] will be executed in [CELL] mode
Formula for member [OH Str Cnt] will be executed in [CELL] mode
Formula for member [Current Rtl] will be executed in [CELL] mode
Essbase needs to retrieve [1017] Essbase Kernel blocks in order to calculate the top dynamically-calculated block.
The Dyn.Calc.Cache for database [Drxxxxxx] can hold a maximum of [75] blocks.
The Dyn.Calc.Cache for database [Drxxxxxx], when full, will result in [allocation from non-Dyn.Calc.Cache memory].
Reading Parameters For Database [Drxxxxxx]
Unable to determine the amount of virtual memory available on the system
Index cache size ==> [1048576] bytes, [128] index pages.
Index page size ==> [8192] bytes.
Using buffered I/O for the index and data files.
Using waited I/O for the index and data files.
Unable to determine the amount of virtual memory available on the system
Data cache size ==> [3145728] bytes, [22] data pages
Data file cache size ==> [0] bytes, [0] data file pages
Performing transaction recovery for database [Drxxxxxx] following an abnormal termination of the server.
Restructuring Database [removedfrompost]
Out of disk space. Cannot create a new [Data] file. [adIndNewFile] aborted
Essbase Internal Logic Error [7333]
Exception error log [C:\HYPERION\ESSBASE\app\removedfrompost\log00002.xcp] is being created...
Exception error log completed -- please contact technical support and provide them with this file
RECEIVED ABNORMAL SHUTDOWN COMMAND - APPLICATION TERMINATING

Answer 1

To avoid all these things as a best practice
we didn't allow dense restructure on the cubes size>30 GB
As an altrnative, we will export the level0 data, clear the DB, and load the new data. After that aggregate the cube to store the data at all the consolidation levels.