Log to file or to database table?
I'm developing a web application which uses MS SQL for various data: including users, user accounts, user licenses, license prices, invoices.
I need to log users' real-time usage of the system, and use that for monthly billing: e.g. log whenever a user gets a specific page/URL and bill users at the end of the month based on the number of pages they fetched.
Should I write these log events into a table in my MS SQL database ?
Should I write these log events into a non-SQL append-only log file?
Should I write these log events into a different log file for each user?
This is not an especially high-volume web site: for example, a maximum of 10,000 users each doing an average of 5 loggable events/day => 50,000 events / day = 30 events / minute = 18,000,000 events / year.
I'm asking because either option seems viable and I don't see whether one has a clear advantage.
The data associated with a billable event is simple, e.g.:
User ID (foreign key relationship to the Users table in SQL)
Date and time
URL of the billable page
Good day Tekken
Tekken
>> I need to log users' real-time usage of the system, and use that for monthly billing: e.g. log whenever a user gets a specific page/URL
The IIS already have a great built-in option of logging every request and response. Therefore it give you all the information that you need. It is highly recommended to use it for most cases, if you have this option and you control the IIS logs. The
default location of the log files is: %SystemDrive%\inetpub\logs\LogFiles
If your system content is based on database
for each page (That is something like: Information Management System, or Content Management System), and you need to point to the database server at
each page anyway, then and probably on then, you can use the same connection that you open to the database, in order to keep the information in the database.
Application like DotNetNuke (which is CMS) logs the information in the database by default, for example.
* If you have to be 100% that you do not lose anything then do not recommend to use your own external system like writing to a file from your application. In this case you better to use the database (but for most cases we do not care if we lose one visit
to a page if the server crush at that point, for example)
* I don't think that there is any need for different log file for each user. most common is to configure the iis to build new log file each day for each application. If you have a specific sub-application in the website then you can configure it with separate
logs.
* There several great free log analyzers for the iis logs. In several cases I prefer to actually move the information from the log file, using bulk insert, into the SQL Server but yet for most cases i prefer first writing it externally from the SQL Server.
Writing to file will not give us to keep ACID and we might lose information in some cases. There is option to use NoSQL for fast logging, as well.
* if you do log into the database, then in most case you should not create indexes, and keep the table simple since it is more important to improve insert than select (you should never use update or delete on logs).
>> This is not an especially high-volume web site: for example, a maximum of 10,000 users each doing an average of 5 loggable events/day =>
50,000 events / day = 30 events / minute = 18,000,000 events / year.
This look like a very small activity and basically I guess (we don't have all the information to suggest) that Database solution should be OK in your case. Make sure that you do not use express edition and you use several CPU in the database, will make much
more reliable for multi threads and more activities on the website.
Ronen Ariely
[Personal Site] [Blog] [Facebook]
Similar Messages
-
Hello Anybody, I have a question. Can any of you please suggest me how to make an xml file from the database table with all the records?
Note:- I am having the XSD Schema file and the resulted XML file should be in that XSD format only.The Oracle documentation has a good overview of the options available
Generating XML Data from the Database
Without knowing your version, I just picked 11.2, so you made need to look for that chapter in the documentation for your version to find applicable information.
You can also find some information in XML DB FAQ -
A query while importing an XML file into a Database Table
Hi,
I am Creating an ODI Project to import an XML file into a Database Table.With the help of the following link
http://www.oracle.com/webfolder/technetwork/tutorials/obe/fmw/odi/odi_11g/odi_project_xml-to-table/odi_project_xml-to-table.htm
I am facing a problem while creating Physical Schema for the XML Source Model.
For the
Schema(Schema)
and
Schema(Work Schema) field they have selected GEO_D.
What is GEO_D here??
or
What I have to select here?
1) schema of the xml file (NB:-I havn't created any .xsd or .dtd file for my .xml file)
or
2)my target servers schema
Please tell me what I'll do??
Thanksand
Schema(Work Schema) field they have selected GEO_D.
What is GEO_D here??This is the schema name which is specified in the XML file .
What I have to select here?
1) schema of the xml file (NB:-I havn't created any .xsd or .dtd file for my .xml file)Yes
2)my target servers schema
Please tell me what I'll do??
Thanks -
Csv file uploading for database table creation
Hi there,
I'm in the process of making an application that will be able to upload a csv file and create a table based on the same file. As of now, I have managed to make my application upload a csv file into the database. My problem now is to transfer the data in the csv into a table. If there is a function that can do this, please let me know. But as of now, I have tried all that I can but in vain. I would appreciate any assistance rendered as to how I can go about this.
Kind regards,
Lusunthahai Lusuntha ,
Go to search forum and type "upload within html db".here u will find the required information ,as well as the code.go for each topic in the search result. -
Dump File Size vs Database Table Size
Hi all!
Hope you're all well. If Datapump estimates that 18 million records will produce a 2.5GB dumpfile, does this mean that 2.5GB will also be consumed on the database table when this dump file is imported into a database?
Many thanks in advance!
Regards
ACdoes this mean that 2.5GB will also be consumed on the database table when this dump file is imported into a database?No since the size after import depends on various factors like block size, block storage parameters etc.
-
Inserting records from a txt file to a database table
I would like to know how to insert records from a file (txt) to a database table through a java application?
[BufferedReader |http://java.sun.com/javase/6/docs/api/java/io/BufferedReader.html] and PreparedStatement
IO and JDBC -
Loading an XML file into a database table.
What is the convenient way to parse XML data present in a data file in server?
Hi;
Please check:
http://riteshkk2000.blogspot.com/2012/01/loadimport-xml-file-into-database-table .html
http://docs.oracle.com/cd/E18283_01/appdev.112/e16659/xdb03usg.htm#BABIFADB
http://www.oracle.com/technetwork/articles/quinlan-xml-095823.html
Regard
Helios -
Uploading Excel file into SAP Database table?
I built a table in the SAP Data Dictionary, and i need to write a program that uploads the Excel table, into the SAP Database table. Does anybody have a sample program that may help me? Thanks!
TYPES:
BEGIN OF ty_upload,
matnr like mara-matnr,
meins like mara-meins,
mtart like mara-mtart,
mbrsh like mara-mbrsh,
END OF ty_upload.
DATA it_upload TYPE STANDARD TABLE OF ty_upload WITH header line.
DATA wa_upload TYPE ty_upload.
DATA: itab TYPE STANDARD TABLE OF alsmex_tabline WITH header line.
CALL FUNCTION 'ALSM_EXCEL_TO_INTERNAL_TABLE'
EXPORTING
filename = 'C:\Documents and Settings\venkatapp\Desktop\venkat.xls'
i_begin_col = 1
i_begin_row = 1
i_end_col = 4
i_end_row = 65535
TABLES
intern = itab.
if not itab[] is initial.
loop at itab .
case itab-col.
when '0001'.
it_upload-matnr = itab-value.
when '0002'.
it_upload-meins = itab-value.
when '0003'.
it_upload-mtart = itab-value.
when '0004'.
it_upload-mbrsh = itab-value.
append it_upload.
clear it_upload.
clear itab.
endcase.
endloop.
endif.
loop at it_upload into wa_upload.
ztable-matnr = wa_upload-matnr.
ztable-meins = wa_upload-meins.
ztable-mtart = wa_upload-mtart.
ztable-mbrsh = wa_upload-mbrsh.
insert ztable.
endloop. -
How do I import the data from Excel (.xls) file into Oracle Database table
I have an excel file with 5 columns of 5000 rows.
I have to import 5000 rows of data into my oracle table with
same column names & Data types.
What is the best advise for this ?
Thanks.
SriniYou can connect to Oracle from Excel directly. I know you can read data but am not sure if you can export data.
Other option is to import the data into Access.
Create an ODBC connection for Oracle
Use the to connect to Oracle from Access
Export data to Oracle from Access
Shakti
http://www.impact-sol.com
Developers of Guggi Oracle - Tool for DBAs and Developers -
How I store my sound file in my database table
Dear friend
i make a table for store sound file
table is create .
but now
how i insert that file which is at 'c:\my document '
to this table.
i try following command but it has a error
insert into sound values('c:\my document\abc.au');
what i do for it
please mail me as soon as on following id
[email protected]
ok byeDear friend
i make a table for store sound file
table is create .
but now
how i insert that file which is at 'c:\my document '
to this table.
i try following command but it has a error
insert into sound values('c:\my document\abc.au');
what i do for it
please mail me as soon as on following id
[email protected]
ok bye -
Can WS Audit logs be send to a database instead than to a file?
Is it possible to configure OC4J so that the audit logs are sent to a database table instead to the log.xml file? I'm trying to write an admin console through the web where the user can see the failed invocations to the webservice. It would be nice if I'm able to read the messages processed from a database table.
Any suggestions?
Thanks,
Mario
Edited by: user521348 on Nov 3, 2009 3:21 AMYes, OAS does.
Please check the documentation about this feature in OAS Administration documentation. -
What's the best way to create XML file with a schema from a database table?
Hi,
I want to create an XML file from a database table (which has over 600 columns) using XML schema (.xsd file). I want to know the best way to do this.
The output XML file is NOT a direct data dump from the DB table, there�re some logic around it, such as the XML file has some hierarchy with repeating tags.
I have done this using JAXB by creating Java classes form XML schema, but I don�t want to map 600 DB columns to these Java classes manually, and loop through the record set to create repeating tags.
I know there are few tools around now like MapForce (Altova), how do people do these now?
Thanks,
ChandiCan you use a schema when we compose XML doc from Database tables?
Actually, I'm using SQL Server (sorry, wrong forum). But, I thought a Java tool would have a solution for me. -
Using Dense/Sparse Lookup on Tables or files from different Databases
Hi,
I have a Subject Area which is built on Essbase cube. I have a requirment where I have to use a Lookup on Flat file and/or Database Table.
I followed below steps
1. Created a Logical Column in BMM Layer for a cube dimension
2. Opened the LTS Properties --> Column Mapping and gave below formula
Lookup(SPARSE "Function1"."E:\502125500".""."Function2.txt"."ALIAS", 'NA', "TNWD06613.genpitfi01.og.ge.com"."GOGREP".""."GOGREP"."Gen5,Functions" )
where
Function1"."E:\502125500".""."Function2.txt"."ALIAS" ---> Column from File (Created File DSN and imported in RPD)
"TNWD06613.genpitfi01.og.ge.com"."GOGREP" ---> Essbase Cube
3. Dragged the Column to Presentation for Subject Area on Cube
When I try to create a report on this column I get below error.
Error Codes: OPR4ONWY:U9IM8TAC:OI2DL65P
State: HY000. Code: 10058. [NQODBC] [SQL_STATE: HY000] [nQSError: 10058] A general error has occurred. [nQSError: 43113] Message returned from OBIS. [nQSError: 16001] ODBC error state: 22005 code: -3030 message: [Microsoft][ODBC Text Driver] Expression too complex. [nQSError: 16015] SQL statement execution failed. (HY000)
SQL Issued: SELECT 0 s_0, "Essbase Test"."Functions"."Alias" s_1, "Essbase Test"."Functions"."Gen5,Functions" s_2, SORTKEY("Essbase Test"."Functions"."Gen5,Functions") s_3 FROM "Essbase Test" FETCH FIRST 65001 ROWS ONLY
Can anybody help me on the same?You can script schemas from all DBs and in case of conflicts you will have to decide what to do.
For example if you have the same SP or UDF on two DBs that does different things, you need to decide what to do with them.
Only if the conflict is tables with the same structure and without KEY constraints and you can also script data and go on.
If you have PK-FK relationships (which I bet is the case) you really need to design your changes based on them and I suspect you will end up scripting instead of putting boxes on SSIS.
It really is a migration job for an experienced DBA and am afraid SSIS can do that much as
Integrate.. not Migrate. -
Text file storing in a database table
I have a text file, that contains so many lines in it,..
Now the problem is that.. I want to store my text file with the pl/sql code in a database table....
plz suggest, how to create a table and how to store that text file in a database table.....The following code is tested and get the correct result (forms 10.1.2.0):
Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> create table tclob (col_lob clob);
Table created.
Declare
file TEXT_IO.FILE_TYPE;
text varchar2(10000);
L$Clob CLOB;
Begin
-- prepare CLOB object --
dbms_lob.createtemporary( L$Clob, TRUE ) ;
-- open text file --
file := TEXT_IO.FOPEN( 'c:\forms.css', 'R' ) ;
If TEXT_IO.IS_OPEN( file) Then
-- start reazding file --
Begin
Loop
TEXT_IO.GET_LINE( file, text );
-- add the line to the CLOB variable --
dbms_lob.writeappend (L$Clob, length(text), text);
End loop;
Exception
when no_data_found then
-- end of text file --
TEXT_IO.FCLOSE(file);
when others then
null;
End ;
-- store the CLOB in the database --
--Update tclob Set col_lob = L$Clob ;
insert into tclob (col_lob) values (L$Clob);
forms_ddl('commit');
dbms_lob.freetemporary( L$Clob ) ;
End if ;
exception
when others then
message(error_code || '-' || error_text);
End;Francois -
Dropping log file in standby database
Please,
I need a help for the following issue:
I'm making a technical documentation on various event that occur on dataguard configuraation, right now I drop a redo log group file on primary database, and when I try to drop the equivalent log group file on standby database I got the following error:
SQL> alter database drop logfile group 3;
alter database drop logfile group 3
ERROR at line 1:
ORA-01156: recovery in progress may need access to files
this is the current state of the redolog file on standby database.
SQL> select group#,members,status from v$log;
GROUP# MEMBERS STATUS
1 3 CLEARING_CURRENT
3 3 CLEARING
2 3 CLEARING
Eventhough I do the following command on standby I also get an error.
SQL> ALTER DATABASE CLEAR LOGFILE GROUP 3;
ALTER DATABASE CLEAR LOGFILE GROUP 3
ERROR at line 1:
ORA-01156: recovery in progress may need access to files
Can someone tell me how to drop on dataguard configuration the redolog file on primary database and their corresponding on standby database
I'm working on 10 release 2, on Windows
Thanks youOracle Dataguard Concept and administration release 2,ref b14239: is my source but it doesn't work when trying to drop stanby group or logile member.
For example, if the primary database has 10 online redo log files and the standby
database has 2, and then you switch over to the standby database so that it functions
as the new primary database, the new primary database is forced to archive more
frequently than the original primary database.
Consequently, when you add or drop an online redo log file at the primary site, it is
important that you synchronize the changes in the standby database by following
these steps:
1. If Redo Apply is running, you must cancel Redo Apply before you can change the
log files.
2. If the STANDBY_FILE_MANAGEMENT initialization parameter is set to AUTO,
change the value to MANUAL.
3. Add or drop an online redo log file:
■ To add an online redo log file, use a SQL statement such as this:
SQL> ALTER DATABASE ADD LOGFILE '/disk1/oracle/oradata/payroll/prmy3.log'
SIZE 100M;
■ To drop an online redo log file, use a SQL statement such as this:
SQL> ALTER DATABASE DROP LOGFILE '/disk1/oracle/oradata/payroll/prmy3.log';
4. Repeat the statement you used in Step 3 on each standby database.
5. Restore the STANDBY_FILE_MANAGEMENT initialization parameter and the Redo Apply options to their original states.
Thank
Maybe you are looking for
-
Gcc 4.7.2 fails to find/(or something else with) default library
Hi! I use the gcc compiler (version 4.7.2) but when I try to compile using the following: g++ test.cpp I have verified that the code itself is perfectly valid because the following error does not occur when compiling on another machine (with the same
-
1-I create a check box on the Pick & pack selection screen. I'm using the 'top' property to position my check box at the same horizontal level than the 'expand' button. But the check box appears much higher than I expected. When I check the 'top' pro
-
Dragging and Dropping eBooks in iTunes
I have iTunes 11.0.1 on my MacBook 2,1 that I use to sync my Music to my iPad, third generation. I am told that I can use iTunes to also sync non Apple Store purchased eBooks purchases to my iPad. I would do this by just dragging /dropping the eboo
-
Having problem placing a picutre with script
hi all im kinda new to this i have 1 pic i want to place in specific coordinate i opened up a new document A4 and i opened my script i have a problem with the x and y coordinate the numbers doesnt make sense what i did? i manualy checked for the x an
-
Having problems setting alarm time since daylight savings time change today
Since the change to Daylight Savings time this morning, I am unable to set my alarm correctly. I set it to 2:40 a.m. but the time on the alarm shows 3:40 a.m. I have all the proper settings correctly done. I have powered down the phone numerous ti