How can I improve large deletes?

Question

Hi,
In order to keep our database size to an acceptable level, we delete all records older than 4 hours, every 15 minutes with a delete statement like this:
DELETE FROM mytable WHERE add_time < (1174391100000 - 14400000);
add_time is a BIGINT containting the number of milliseconds when the record was added to "mytable". 1174391100000 is the current time in ms and 14400000 is 4 hours in ms.
This delete removes on average 200000 records from a database with some 2 million records. The delete takes about 30 seconds. The documentation states that it is wise to avoid large deletes. Howvever I can't use TRUNCATE, since most of the data is still in use. The number of records to delete vary, so I don't know how many times a DELETE FIRST 10000 would be needed.
Is there any other way to speed up the delete process in TimesTen?
(e.g. in Oracle you would use a PL/SQL procedure, loop through a cursor and commit every 10000 records).

Answer 1

bwas.as_user schema =
CREATE TABLE bwas.as_user (
user_uid INTEGER not null,
cust_group_uid INTEGER not null,
user_id VARCHAR(161) NOT INLINE NOT NULL,
first_name VARCHAR(30) not null,
last_name VARCHAR(30) not null,
first_name_unicode NVARCHAR(30) not null,
last_name_unicode NVARCHAR(30)not null,
title NVARCHAR(50) NOT INLINE,
email VARCHAR(80) NOT INLINE,
mobile_odn CHAR(30),
pager_odn CHAR(30),
name_rec_flag CHAR(1) not null,
name_rec_filename NVARCHAR(256) NOT INLINE,
time_zone_id VARCHAR(80) NOT INLINE,
auth_failed_flag CHAR(1) not null,
voice_portal_pswd CHAR(30),
user_type CHAR(10) not null,
primary_as_uid INTEGER not null,
active_as_uid INTEGER not null,
dept_uid INTEGER,
hiragana_first_name NVARCHAR(30),
hiragana_last_name NVARCHAR(30),
vp_pswd_not_set_flag CHAR(1),
clid_num CHAR(16),
language_uid INTEGER not null,
yahoo_id VARCHAR(50) NOT INLINE,
failed_vp_attempt TINYINT,
last_passcode_modified BIGINT,
vp_fail_notification_sent CHAR(1),
search_last_name_unicode NVARCHAR(30) not null,
search_first_name_unicode NVARCHAR(30) not null,
vp_auto_login CHAR(1) not null,
PRIMARY KEY (user_uid),
FOREIGN KEY (cust_group_uid) REFERENCES bwas.cust_group (cust_group_uid),
FOREIGN KEY (primary_as_uid) REFERENCES bw.peering_hosts (host_uid),
FOREIGN KEY (active_as_uid) REFERENCES bw.peering_hosts (host_uid),
FOREIGN KEY (dept_uid) REFERENCES bwas.dept (dept_uid),
FOREIGN KEY (language_uid) REFERENCES bwas.language (language_uid))
UNIQUE HASH ON (user_uid) PAGES = 10;
There are 10098 rows in bwas.as_user
bwas.cust_group schema =
CREATE TABLE bwas.cust_group (
cust_group_uid INTEGER not null,
cust_group_id VARCHAR(30) not null,
name VARCHAR(80) NOT INLINE,
name_unicode NVARCHAR(80) NOT INLINE,
svc_provider_uid INTEGER not null,
pswd_rule_uid INTEGER not null,
time_zone_id VARCHAR(80) NOT INLINE NOT NULL,
extension_len TINYINT not null,
clid_num CHAR(16),
use_clid_num_flag CHAR(1) not null,
use_desc_clid_flag CHAR(1) not null,
user_limit INTEGER not null,
loc_code CHAR(15),
search_name_unicode NVARCHAR(80) NOT INLINE,
PRIMARY KEY (cust_group_uid),
FOREIGN KEY (svc_provider_uid) REFERENCES bwas.svc_provider (svc_provider_uid),
FOREIGN KEY (pswd_rule_uid) REFERENCES bwas.pswd_rule (pswd_rule_uid))
UNIQUE HASH ON (cust_group_uid) PAGES = 10;
Number of rows = 53
bwas.svc_provider schema =
CREATE TABLE bwas.svc_provider (
svc_provider_uid INTEGER not null,
svc_provider_id VARCHAR(30) not null,
name NVARCHAR(80) NOT INLINE,
pswd_rule_uid INTEGER not null,
support_email VARCHAR(80) NOT INLINE,
enterprise_flag CHAR(1) NOT NULL,
use_custom_routing_prof_flag CHAR(1) NOT NULL,
PRIMARY KEY (svc_provider_uid),
FOREIGN KEY (pswd_rule_uid) REFERENCES bwas.pswd_rule(pswd_rule_uid))
UNIQUE HASH ON (svc_provider_uid) PAGES = 10;
Number of rows = 4
The delete statement I posted initially wasn't quite correct, here is the correct one:
DELETE FIRST 1000
FROM bwas.user_assn_ftr_svc
WHERE user_uid
IN (SELECT user_uid
FROM bwas.as_user,
bwas.cust_group,
bwas.svc_provider
WHERE as_user.cust_group_uid = cust_group.cust_group_uid
AND cust_group.svc_provider_uid = svc_provider.svc_provider_uid
AND svc_provider.svc_provider_id = ? );
The query plan for the select is:
Query Optimizer Plan:
STEP: 1
LEVEL: 3
OPERATION: RowLkTtreeScan
TBLNAME: BWAS.SVC_PROVIDER
IXNAME: IDXUSVC_PROVIDER01
PRED: SVC_PROVIDER.SVC_PROVIDER_ID = 'MarioSP50'
OTHERPRED: <NULL>
STEP: 2
LEVEL: 3
OPERATION: RowLkTtreeScan
TBLNAME: BWAS.CUST_GROUP
IXNAME: TTFOREIGN_11
PRED: CUST_GROUP.SVC_PROVIDER_UID = SVC_PROVIDER.SVC_PROVIDER_UID
OTHERPRED: <NULL>
STEP: 3
LEVEL: 2
OPERATION: NestedLoop
TBLNAME: <NULL>
IXNAME: <NULL>
PRED: <NULL>
OTHERPRED: <NULL>
STEP: 4
LEVEL: 2
OPERATION: RowLkTtreeScan
TBLNAME: BWAS.AS_USER
IXNAME: TTFOREIGN_34
PRED: AS_USER.CUST_GROUP_UID = CUST_GROUP.CUST_GROUP_UID
OTHERPRED: <NULL>
STEP: 5
LEVEL: 1
OPERATION: NestedLoop
TBLNAME: <NULL>
IXNAME: <NULL>
PRED: <NULL>
OTHERPRED: <NULL>
Here are the ODBC settings we use on the Solaris system:
Driver=/usr/local/TimesTen/tt_base/lib/libtten.so
Authenticate=0
AutoCreate=0
DurableCommits=0
ExclAccess=0
Isolation=1
LockLevel=0
Logging=1
LogBuffSize=8192
LogFileSize=8
LogPurge=1
MatchLogOpts=0
MemoryLock=0
OverWrite=0
PermSize=512
Preallocate=0
Temporary=0
TempSize=170
ThreadSafe=1
WaitForConnect=0