Data_Transfer and Database size.
Hello all,
I am using a Data_Transfer transform to speed up job performance. My question is that if I use these transforms will database size increase by a larger percentage than if not using it. And if so are there options that I can select to limit the size in growth in Data Services and in the Database? The database that is being used is Oracle 11g.
Thank you
Hi Shiva,
Thank you for your response. Just to make sure I understand you mean that after the job is done, whatever temp tables are creating during the "Data_Transfer" are removed from the database and will not take up space? Also does using this method increase the database log size much more than just using a DS job without it?
Thank you and sorry for all the questions.
Similar Messages
-
Sharepoint Site and Database Size Conflict
Hello,
We are at the initial face of migration from Sharepoint 2010 to Sharepoint online. We have gather all the reports but for some reason, Report for Site show more data size than Content database.
eg. I have content Database size of 5 GB.
Site data which also include the subsite size total show as 13 GB. I am confused how is that possible ? Any help appreciatedHi Priti, see the following links for the reason:
https://social.msdn.microsoft.com/Forums/sharepoint/en-US/34ad8581-b4ae-442d-bce5-1a1f72f6e472/sharepoint-site-and-content-database-size-discrepency?forum=sharepointgenerallegacy
https://social.msdn.microsoft.com/Forums/sharepoint/en-US/6b5c33f8-be08-466f-81aa-359f8a03ede5/sharepoint-site-collection-size-vs-database-properties-sizemdf-file-size?forum=sharepointadminprevious
cameron rautmann -
Transaction Sync and Database Size
Hello,
We're using BDB (via the Java interface) as the persistent store for a messaging platform. In order to achieve high performance, the transactional operations are configured to not sync, i.e., TransactionConfig.setSync(false) . While we do achieve better performance, the size of the database does seem rather large. We checkpoint on a periodic basis, and each time we checkpoint, the size of the database grows, even though records (messages in our world) are being deleted. So, if I were to add, say 10000 records, delete all of them and then checkpoint, the size of the database would actually grow! In addition, the database file, while being large, is also very sparse - a 30GB file when compressed reduces in size to 0.5 GB.
We notice that if we configure our transactional operations to sync, the size is much smaller, and stays constant, i.e., if I were to insert and subsequently delete 10000 records into a database whose file is X MB, the size of the database file after the operations would be roughly X MB.
I understand that transaction logs are applied to the database when we checkpoint, but should I be configuring the behaviour of the checkpointing (via CheckpoinConfig )?
Also, I am checkpointing periodically from a separate thread. Does BDB itself spawn any threads for checkpointing?
Our environment is as follows:
RedHat EL 2.6.9-34.ELsmp
Java 1.5
BDB 4.5.20
Thanks much in advance,
PrashanthHi Prashanth,
If your delete load is high, your workload should benefit from setting the DB_REVSPLITOFF flag, which keeps around the structure of the btree regardless of records being deleted. The result should be less splits and merges, and is therefore better concurrency.
Here you can find some documentation that should help you:
Access method tuning: http://www.oracle.com/technology/documentation/berkeley-db/db/ref/am_misc/tune.html
Transaction tuning: http://www.oracle.com/technology/documentation/berkeley-db/db/ref/transapp/tune.html
If you are filling the cache with dirty pages, you can indeed call checkpoint() periodically in the application, or you can create a memp_trickle thread. See the following sections of the documentation:
- Javadoc: Environment -> trickleCacheWrite" http://www.oracle.com/technology/documentation/berkeley-db/db/java/com/sleepycat/db/Environment.html#trickleCacheWrite(int)
Some related thread for the "database size issue", can be found here: http://forums.oracle.com/forums/thread.jspa?threadID=534371&tstart=0
Bogdan Coman -
Rman restore and database size
I am testing a production database backup from tape.
Our test machine does not have the same disk space as our production machine.
Right now, here is the production size info:
Allocated (GB) 137.16
Used (GB) 60.71
Allocated Free Space (GB) 76.45
Is there a way to restore with rman and restore only the actual size of the database?
In my case 61gig instead of 140G.Then i am confuse. I had this other problem where i had a big aud$ table in my system tablesace. I have truncated the table, now i need my 20gig space back.
auditing
The solution proposed was to do a backup and restore using RMAN after truncating AUD$:
http://www.dizwell.com/prod/node/613
I did not tried it yet. But after reading this, i though i could be able to restore a rman backup that takes less space. -
Database size and Storage Metrics are mismatching
Hi,
When I look at the storage metrics for one of our SharePoint Web Application root site collection it shows the top sized site as 28.2% of parent and the size as 21178.5 MB. When we calculate the total size of the site collection from
that it should 74994 MB. But when I see the size of the content database ( this is the only site collection in this content database) it shows 185661.5 MB as the size and 26852.5 MB as the space available. When I run the report (Disk Usage By Top Tables)
on that database it brings the list with dbo.AllDocStreams table at the top (Data - 80333 MB) followed by dbo.AuditData (Data - 48305 MB) followed by dbo.AllUserData (Data - 8593 MB).
3 years back we migrated this SharePoint Web Application from 2007 to 2010.
I saw that the recycle bin has only 30 day worth data.
How can we reduce the size of the database according to what it shows in the storage metrics? Appreciate any help.
Thanks.
raniso... your database has:
80gb in files (includes version history)
50gb in audit data
8.5gb of list data
plus ~27gb of free space for growth between backups
total of ~165gb (which is close enough to the 185661.5 that you listed)
you need to split up the site collection, and then spread the site collections across multiple databases.
Scott Brickey
MCTS, MCPD, MCITP
www.sbrickey.com
Strategic Data Systems - for all your SharePoint needs -
Counting the size of tablespaces and database
hi,
can any one help me with this..
to write a shell script that counts the size of the tablespaces (excluding the oracle tablespaces like system,sysaux) and come up with the individual database size of evrey databse and mail to team on every month of 2nd for each database.
thanks in advance..to write a shell script that counts the size of the tablespaces (excluding the oracle tablespaces like system,sysaux)To do so requires one to know which shell you use.
Since you declined to share that with us, You're On Your Own YOYO)!
How does script know/decide which DBs exist on any system?
Handle: user8884944
Status Level: Newbie
Registered: Dec 13, 2009
Total Posts: 23
Total Questions: 15 (15 unresolved)
Why are you so challenged to get answers to your posts?
Edited by: sb92075 on Jan 6, 2010 4:52 PM -
I have deleted a large number of records from a database which accounted for much of the space being used as it was not needed at this time. When I take a backup of the database will the size of the db I then restore from this backup be reduced or should
I shrink the database first. Didn't want to have to perform a shrink operation if not necessary as I've seen posts that it affects performance.
Thanks
RobertIf you have deleted chunks of data and take the backup the size of the backup will be reduced. However if you want to release the unused space from the running database(with out backup and restore), you could do a shrink
Have a look at this link
http://dba.stackexchange.com/questions/28360/sql-server-database-size-didnt-decrease-after-deleting-large-number-of-rows
from
the above link
I've blogged about this where I admonished folks to "Don't
touch that shrink button!" but sometimes... Sometimes you need to.
Satheesh
My Blog |
How to ask questions in technical forum -
Hi All,
Environment: SharePoint 2010 with SP2.
Issue: One of the Content databases size in our farm shows 200 GB as used. There are 25 site collections in the DB and we counted
the total storage used of all site collections in that Content DB and is not more than 40 GB. (used "enumsites" command and counted the each sitecollections storage used).
What actions/troubleshooting were done?
Ran one script which will find the actual size of site collection and how much space used in the disk. But didn’t find major difference in the report.
Checked “Deleted from end user recycle bin “in all the site collections and no major storage is noticed.
Planning to Detach the problematic Content DB and attach ,will check whether any major effect is there
Why the Content DB shows 200 GB as used when the total storage used of all site collections is just 40 GB.
Appreciate suggestions from any one.
Best Regards,
Pavan Kumar Sapara.
s p kumarHi,
Thanks for your reply.
As there is only 20 MB of unallocated space for the above said DB, SQL DB team informed that they cannot perform the DB shrink at this moment.
So we are thinking to Offload all the site collections to other new DB and then will Drop the problematic database. In this way we can overcome the
issue.
Answer for your queries.
Are the mismatched sizes causing an issue? Are you short on diskspace for DB storage or SQL backups?
No, there is no issue with the mismatched sizes. We are not on short on disk space. We just worried why it occupies that much size(200 GB) when
the total storage used of all site collections in that DB is 40 GB.
Best Regards,
Pavan Kumar Sapara.
s p kumar -
How can we see what is the CPU and RAM size for the database?
<div class="jive-quote">select * from v$version</div>
BANNER
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bi
PL/SQL Release 10.2.0.5.0 - Production
CORE 10.2.0.5.0 Production
TNS for HPUX: Version 10.2.0.5.0 - Production
NLSRTL Version 10.2.0.5.0 - Productioncat /proc/meminfo
root: total: used: free: shared: buffers: cached:
Mem: 1055760384 1041887232 13873152 0 100417536 711233536
Swap: 1077501952 8540160 1068961792
MemTotal: 1031016 kB
MemFree: 13548 kB
MemShared: 0 kB
Buffers: 98064 kB
Cached: 692320 kB
SwapCached: 2244 kB
Active: 563112 kB
Inact_dirty: 309584 kB
Inact_clean: 79508 kB
Inact_target: 190440 kB
HighTotal: 130992 kB
HighFree: 1876 kB
LowTotal: 900024 kB
LowFree: 11672 kB
SwapTotal: 1052248 kB
SwapFree: 1043908 kB
Committed_AS: 332340 kB
cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 31
model name : AMD Athlon(tm) 64 Processor 3000+
stepping : 0
cpu MHz : 994.927
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce (snipped)
bogomips : 1956.97
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp -
Increase Database Size popup and never stop
When I opened an link from gmail in safari. The dialog poped up. It said Increase Database Size? Do you want to allow "http://help.apple.com" to use up to 10MB of storage on your iPhone? No matter I choose 'cancel' or 'Increase', it will pop up again. It's so annoying. I happened since two days before.
Who can tell me how to stop it?The same thing is hapening to me. It is very irritating. Could someone please answer how to stop this, without suggesting switching to iMail. I have looked all over the web and lots of people asking about about this, but nobody offering the solution.
Kevin -
Command of database used and free size
Good Day for all of you .
i need command to check whole database used and free size and table spaces , datafile used and free size.
your help and cooperation highly appreciated.Query 1:
select case grouping(Tablespace_Name)
when 0 then Tablespace_Name when 1 then '======> TOTAL <======' end Tablespace_Name,
autoextensible AutExt,trunc((sum(Bytes)/1024/1024),2)MB_SPC_OCCUP,
trunc((decode(sum(MaxBytes),0,sum(bytes),sum(MaxBytes))/1024/1024),2) MB_TOT_AVAIL,
trunc((decode(sum(MaxBytes),0,0,sum(MaxBytes)-sum(Bytes))/1024/1024),2) MB_Free_Spc,
trunc(decode(sum(MaxBytes),0,0,
(sum(MaxBytes)- sum(Bytes))/sum(MaxBytes)*100),2)Pct_Free_Spc,
status from dba_data_files group by cube(Tablespace_Name), Autoextensible, status
order by status,autoextensible,Tablespace_Name desc;
Query 2:
select (select decode(extent_management,'LOCAL','*',' ') from dba_tablespaces where tablespace_name
= b.tablespace_name)
|| nvl(b.tablespace_name, nvl(a.tablespace_name,'UNKOWN')) name,
kbytes_alloc kbytes,
kbytes_alloc-nvl(kbytes_free,0) used,
nvl(kbytes_free,0) free,
((kbytes_alloc-nvl(kbytes_free,0))/kbytes_alloc)*100 pct_used,
nvl(largest,0) largest,
nvl(kbytes_max,kbytes_alloc) Max_Size,
decode(kbytes_max, 0, 0, (kbytes_alloc/kbytes_max)*100) pct_max_used
from ( select sum(bytes)/1024 Kbytes_free,
max(bytes)/1024 largest,
tablespace_name
from sys.dba_free_space
group by tablespace_name ) a,
( select sum(bytes)/1024 Kbytes_alloc,
sum(maxbytes)/1024 Kbytes_max,
tablespace_name
from sys.dba_data_files
group by tablespace_name
union all
select sum(bytes)/1024 Kbytes_alloc,
sum(maxbytes)/1024 Kbytes_max,
tablespace_name
from sys.dba_temp_files
group by tablespace_name )b
where a.tablespace_name (+) = b.tablespace_name
Query 3:
select b.tablespace_name, tbs_size SizeMb, a.free_space FreeMb
from
(select tablespace_name, round(sum(bytes)/1024/1024 ,2) as free_space
from dba_free_space group by tablespace_name) a,
(select tablespace_name, sum(bytes)/1024/1024 as tbs_size
from dba_data_files group by tablespace_name
UNION
select tablespace_name, sum(bytes)/1024/1024 tbs_size
from dba_temp_files
group by tablespace_name ) b
where a.tablespace_name(+)=b.tablespace_name
Query 4:
col "Tablespace" for a22
col "Used MB" for 99,999,999
col "Free MB" for 99,999,999
col "Total MB" for 99,999,999
select df.tablespace_name "Tablespace",
totalusedspace "Used MB",
(df.totalspace - tu.totalusedspace) "Free MB",
df.totalspace "Total MB",
round(100 * ( (df.totalspace - tu.totalusedspace)/ df.totalspace))
"Pct. Free"
from
(select tablespace_name,
round(sum(bytes) / 1048576) TotalSpace
from dba_data_files
group by tablespace_name) df,
(select round(sum(bytes)/(1024*1024)) totalusedspace, tablespace_name
from dba_segments
group by tablespace_name) tu
where df.tablespace_name = tu.tablespace_name ;
Query 5:
select t1.tablespace_name,
sum(decode(t1.autoextensible, 'NO', t1.bytes, 'YES', t1.maxbytes))/1024 max_size,
sum(nvl(t2.free, 0) + decode(t1.autoextensible, 'NO', 0 , 'YES', t1.maxbytes -
t1.bytes))/1024 max_free,
sum(t1.bytes)/1024 current_size,
sum(nvl(t2.free, 0))/1024 current_free
from (select file_id, file_name, tablespace_name, bytes, maxbytes, autoextensible from
dba_data_files) t1
left outer join
(select file_id, sum(bytes) free from dba_free_space group by file_id) t2
on t1.file_id = t2.file_id
group by tablespace_name
You can choose from any one or more as your requirement / choice.
Regards
Girish -
Get DataBase Size and Compare it
Hello Folks
I'm trying to get SQL 2008 R2 database size and compare it with this code:
$DBSize = Invoke-SQLCmd -Query "sp_databases" -Username "user" -Password "pwd" `
-ServerInstance "localhost" | where-object {$_.database_name -eq "ABC"} | select database_size
If(($DBSize | Out-String) -ge "5242880"){
# Some action #
But that don't return nothing... Some help !?
Jimcesse
Principal: http://sysadmin-cr.com/
Alterno: http://blogs.itpro.es/jimcesseHi,
I have no way to test this, but try adding -ExpandProperty to select.
Also, I suggest getting rid of Out-String and using numbers instead.
EDIT: Also, you may need to use a loop to check each returned size if the query is returning more than one database size.
Don't retire TechNet! -
(Don't give up yet - 13,085+ strong and growing) -
SQL azure database size not dropping down after deleting all table
Dear all,
I have a simple database on Azure for which I have deleted all table data. The size of the database is still showing 5Mb of data and I am charge for that. I have heard that this may happen from cluster index getting fragmented.
I have run a querry I found on internet on all my table index to show percentage of fragmentation and all report 0%.
DBA is not so my job but what could it be or how can I reduce that size ?
ON premise I would use COMPACT DB but not available in azure like some others DB action
Thnaks for tips
regardsuser created objects/data are not the only ones stored in your database. you got system objects and metadata as Mike mentions above.
are you trying to skip being charged if you're not storing data? looking at the pricing table, you'll still get charged the $4.995 for the 0-100MB database size range. -
SAP and Database Encrption on SQL Server 2005 or 2008
Hi. I searched as best I could for any existing threads on this topic, and could not find any, so I hope it's OK to raise a new one.
We're running SAP ECC 6.0 on a SQL 2005 database. Database size is around 3.3 TB, with Trans Log of about 550GB, storage is HP EVA.
I've been asked to report on how Database Encryption would impact our systems, both from the perspective of enabling it initially and then how it would affect performance going forward.
I've read a few White Papers, for example:
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/4ab89e84-0d01-0010-cda2-82ddc3548c65?QuickLink=index&overridelayout=true
.. and from what I'm reading, it looks like I would want to be moving to SQL Server 2008 as a first step. Does anyone have any experience of implementing SQL encryption for a SAP database, either in 2K5 or 2K8?
All comments or help will be welcomed.
Thanks
Steven ClarkHello Steven,
You can also check the following websites:
http://blogs.msdn.com/b/saponsqlserver/archive/2011/09/05/decreased-disk-space-requirement-for-sap-unicode-systems.aspx
http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/50c5de8f-282a-2e10-cb8a-a5859a0994a4&overridelayout=true
And, SAP notes:
[1581700 PAGE compression support for DDIC & Hom./Het. System Copy|http://service.sap.com/sap/support/notes/1581700]
[1488135 Database compression for SQL Server|http://service.sap.com/sap/support/notes/1488135]
Regards,
Eduardo Rezende -
Tables are deleted but database size does not change in sql server 2008r2
Hi All,
20GB Tables are deleted in my database but database size does not change and disk size showing same size.Hi ,
I have ran the Disk usage by Top Tables report and Identified couple of tables with unwanted data for last 5 years. I have deleted the data for the first 3 years and then ran the Disk usage by Top Tables report again. When I compared the report before
and after the data deletion, I have noticed certain facts which is not matching with what I know or learned from experts like you. The following are the points where I am looking for clarification:
1.Reserved (KB) has been reduced. I was expecting the data Reserved (KB) will remain the same after the data deletion.
2. The Data(KB) and Indexes(KB) fields have been reduced as expected. The Unused(KB) field have been increased as expected.
I was expecting the total of Data(KB) and Indexes(KB) field space gained will be equal to the Unused(KB) field gained after deleting the data. But that is not the case. When I deducted(subtracted) the difference in the Reserved(KB)(Difference before
and after data deletion) field from the Total of space gained from the data deletion is equal to the Unused(KB) gained field value.
I am not a SQL expert and not questioning but trying to understand whether we really gain space by deleting data from the tables. Also keen to get the concepts right, but my testing by deleting some records confused me.
Looking ahead to all your expert advice.
Thanks,
Vennayat
Maybe you are looking for
-
I do not want to have to choose these two controls or tools in InDesign each time I open a new page or doc. How can I make them open automatically?
-
How to handle comma in the string
I need compare a string in HTMLDB report, as Select HOST, MODEL From hwrpt WHERE MODEL= :P3_MODEL Since most IBM models has 'IBM, XXXX-YYY' syntax for P3:_MODEL, the query failed. I tried to use single/double quoe but failed too. Is there a function/
-
XOR (Exclusive OR logical operator)
Does such a thing exist in SAP ? Do I have something else other than AND and OR as logical operators (like XOR in other languages ? bitwise operators won't help, since I need them in queries). Thanks Avraham
-
Blanket Fill in for same field on multiple page form?
I have a form that is many pages long and it has certain fields that appear on each page... (ie Name, address etc) Anyone know of a way for the user to fill in the field on page 1 and the other fields will automatically populate on the other pages in
-
Where can I find the iOS 6 to download
I restored an iPad, and it only comes with iOS 5.1. Under settings it will not offer iOS 6, and I can not get it from apps or apple.com . I ultimately want to download iOS 7, but can' t w/ o iOS 6. Thank you.