Log to file or to database table?

I'm developing a web application which uses MS SQL for various data: including users, user accounts, user licenses, license prices, invoices.
I need to log users' real-time usage of the system, and use that for monthly billing: e.g. log whenever a user gets a specific page/URL and bill users at the end of the month based on the number of pages they fetched.
Should I write these log events into a table in my MS SQL database ?
Should I write these log events into a non-SQL append-only log file?
Should I write these log events into a different log file for each user?
This is not an especially high-volume web site: for example, a maximum of 10,000 users each doing an average of 5 loggable events/day => 50,000 events / day = 30 events / minute = 18,000,000 events / year.
I'm asking because either option seems viable and I don't see whether one has a clear advantage.
The data associated with a billable event is simple, e.g.:
User ID (foreign key relationship to the Users table in SQL)
Date and time
URL of the billable page

Good day Tekken
Tekken
>> I need to log users' real-time usage of the system, and use that for monthly billing: e.g. log whenever a user gets a specific page/URL
The IIS already have a great built-in option of logging every request and response. Therefore it give you all the information that you need. It is highly recommended to use it for most cases, if you have this option and you control the IIS logs. The
default location of the log files is: %SystemDrive%\inetpub\logs\LogFiles
If your system content is based on database
for each page (That is something like: Information Management System, or Content Management System), and you need to point to the database server at
each page anyway, then and probably on then, you can use the same connection that you open to the database, in order to keep the information in the database.
Application like DotNetNuke (which is CMS) logs the information in the database by default, for example.
* If you have to be 100% that you do not lose anything then do not recommend to use your own external system like writing to a file from your application. In this case you better to use the database (but for most cases we do not care if we lose one visit
to a page if the server crush at that point, for example)
* I don't think that there is any need for different log file for each user. most common is to configure the iis to build new log file each day for each application. If you have a specific sub-application in the website then you can configure it with separate
logs.
* There several great free log analyzers for the iis logs. In several cases I prefer to actually move the information from the log file, using bulk insert, into the SQL Server but yet for most cases i prefer first writing it externally from the SQL Server.
Writing to file will not give us to keep  ACID and we might lose information in some cases. There is option to use NoSQL for fast logging, as well.
* if you do log into the database, then in most case you should not create indexes, and keep the table simple since it is more important to improve insert than select (you should never use update or delete on logs).
>> This is not an especially high-volume web site: for example, a maximum of 10,000 users each doing an average of 5 loggable events/day =>
50,000 events / day = 30 events / minute = 18,000,000 events / year.
This look like a very small activity and basically I guess (we don't have all the information to suggest) that Database solution should be OK in your case. Make sure that you do not use express edition and you use several CPU in the database, will make much
more reliable for multi threads and more activities on the website.
  Ronen Ariely
 [Personal Site]    [Blog]    [Facebook]

Similar Messages

  • Hello Anybody, I have a question. Can any of you please suggest me how to make an xml file from the database table with all the rows? Note:- I am having the XSD Schema file and the resulted XML file should be in that XSD format only.

    Hello Anybody, I have a question. Can any of you please suggest me how to make an xml file from the database table with all the records?
    Note:- I am having the XSD Schema file and the resulted XML file should be in that XSD format only.

    The Oracle documentation has a good overview of the options available
    Generating XML Data from the Database
    Without knowing your version, I just picked 11.2, so you made need to look for that chapter in the documentation for your version to find applicable information.
    You can also find some information in XML DB FAQ

  • A query while  importing  an XML file into a Database Table

    Hi,
    I am Creating an ODI Project to import an XML file into a Database Table.With the help of the following link
    http://www.oracle.com/webfolder/technetwork/tutorials/obe/fmw/odi/odi_11g/odi_project_xml-to-table/odi_project_xml-to-table.htm
    I am facing a problem while creating Physical Schema for the XML Source Model.
    For the
    Schema(Schema)
    and
    Schema(Work Schema) field they have selected GEO_D.
    What is GEO_D here??
    or
    What I have to select here?
    1) schema of the xml file (NB:-I havn't created any .xsd or .dtd file for my .xml file)
    or
    2)my target servers schema
    Please tell me what I'll do??
    Thanks

    and
    Schema(Work Schema) field they have selected GEO_D.
    What is GEO_D here??This is the schema name which is specified in the XML file .
    What I have to select here?
    1) schema of the xml file (NB:-I havn't created any .xsd or .dtd file for my .xml file)Yes
    2)my target servers schema
    Please tell me what I'll do??
    Thanks

  • Csv file uploading for database table creation

    Hi there,
    I'm in the process of making an application that will be able to upload a csv file and create a table based on the same file. As of now, I have managed to make my application upload a csv file into the database. My problem now is to transfer the data in the csv into a table. If there is a function that can do this, please let me know. But as of now, I have tried all that I can but in vain. I would appreciate any assistance rendered as to how I can go about this.
    Kind regards,
    Lusuntha

    hai Lusuntha ,
    Go to search forum and type "upload within html db".here u will find the required information ,as well as the code.go for each topic in the search result.

  • Dump File Size vs Database Table Size

    Hi all!
    Hope you're all well. If Datapump estimates that 18 million records will produce a 2.5GB dumpfile, does this mean that 2.5GB will also be consumed on the database table when this dump file is imported into a database?
    Many thanks in advance!
    Regards
    AC

    does this mean that 2.5GB will also be consumed on the database table when this dump file is imported into a database?No since the size after import depends on various factors like block size, block storage parameters etc.

  • Inserting records from a txt file to a database table

    I would like to know how to insert records from a file (txt) to a database table through a java application?

    [BufferedReader |http://java.sun.com/javase/6/docs/api/java/io/BufferedReader.html] and PreparedStatement
    IO and JDBC

  • Loading an XML file into a database table.

    What is the convenient way to parse XML data present in a data file in server?

    Hi;
    Please check:
    http://riteshkk2000.blogspot.com/2012/01/loadimport-xml-file-into-database-table .html
    http://docs.oracle.com/cd/E18283_01/appdev.112/e16659/xdb03usg.htm#BABIFADB
    http://www.oracle.com/technetwork/articles/quinlan-xml-095823.html
    Regard
    Helios

  • Uploading Excel file into SAP Database table?

    I built a table in the SAP Data Dictionary, and i need to write a program that uploads the Excel table, into the SAP Database table.  Does anybody have a sample program that may help me?  Thanks!

    TYPES:
         BEGIN OF ty_upload,
         matnr like mara-matnr,
         meins like mara-meins,
         mtart like mara-mtart,
         mbrsh like mara-mbrsh,
         END OF ty_upload.
      DATA it_upload TYPE STANDARD TABLE OF ty_upload WITH header line.
      DATA wa_upload TYPE ty_upload.
      DATA: itab TYPE STANDARD TABLE OF alsmex_tabline WITH header line.
      CALL FUNCTION 'ALSM_EXCEL_TO_INTERNAL_TABLE'
        EXPORTING
          filename    = 'C:\Documents and Settings\venkatapp\Desktop\venkat.xls'
          i_begin_col = 1
          i_begin_row = 1
          i_end_col   = 4
          i_end_row   = 65535
          TABLES
          intern      = itab.
    if not itab[] is initial.
    loop at itab .
    case itab-col.
    when '0001'.
    it_upload-matnr = itab-value.
    when '0002'.
    it_upload-meins = itab-value.
    when '0003'.
    it_upload-mtart = itab-value.
    when '0004'.
    it_upload-mbrsh = itab-value.
    append it_upload.
    clear it_upload.
    clear itab.
    endcase.
    endloop.
    endif.
    loop at it_upload into wa_upload.
    ztable-matnr = wa_upload-matnr.
    ztable-meins = wa_upload-meins.
    ztable-mtart = wa_upload-mtart.
    ztable-mbrsh = wa_upload-mbrsh.
    insert ztable.
    endloop.

  • How do I import the data from Excel (.xls) file into Oracle Database table

    I have an excel file with 5 columns of 5000 rows.
    I have to import 5000 rows of data into my oracle table with
    same column names & Data types.
    What is the best advise for this ?
    Thanks.
    Srini

    You can connect to Oracle from Excel directly. I know you can read data but am not sure if you can export data.
    Other option is to import the data into Access.
    Create an ODBC connection for Oracle
    Use the to connect to Oracle from Access
    Export data to Oracle from Access
    Shakti
    http://www.impact-sol.com
    Developers of Guggi Oracle - Tool for DBAs and Developers

  • How I store my sound file in my database table

    Dear friend
    i make a table for store sound file
    table is create .
    but now
    how i insert that file which is at 'c:\my document '
    to this table.
    i try following command but it has a error
    insert into sound values('c:\my document\abc.au');
    what i do for it
    please mail me as soon as on following id
    [email protected]
    ok bye

    Dear friend
    i make a table for store sound file
    table is create .
    but now
    how i insert that file which is at 'c:\my document '
    to this table.
    i try following command but it has a error
    insert into sound values('c:\my document\abc.au');
    what i do for it
    please mail me as soon as on following id
    [email protected]
    ok bye

  • Can WS Audit logs be send to a database instead than to a file?

    Is it possible to configure OC4J so that the audit logs are sent to a database table instead to the log.xml file? I'm trying to write an admin console through the web where the user can see the failed invocations to the webservice. It would be nice if I'm able to read the messages processed from a database table.
    Any suggestions?
    Thanks,
    Mario
    Edited by: user521348 on Nov 3, 2009 3:21 AM

    Yes, OAS does.
    Please check the documentation about this feature in OAS Administration documentation.

  • What's the best way to create XML file with a schema from a database table?

    Hi,
    I want to create an XML file from a database table (which has over 600 columns) using XML schema (.xsd file). I want to know the best way to do this.
    The output XML file is NOT a direct data dump from the DB table, there�re some logic around it, such as the XML file has some hierarchy with repeating tags.
    I have done this using JAXB by creating Java classes form XML schema, but I don�t want to map 600 DB columns to these Java classes manually, and loop through the record set to create repeating tags.
    I know there are few tools around now like MapForce (Altova), how do people do these now?
    Thanks,
    Chandi

    Can you use a schema when we compose XML doc from Database tables?
    Actually, I'm using SQL Server (sorry, wrong forum). But, I thought a Java tool would have a solution for me.

  • Using Dense/Sparse Lookup on Tables or files from different Databases

    Hi,
    I have a Subject Area which is built on Essbase cube. I have a requirment where I have to use a Lookup on Flat file and/or Database Table.
    I followed below steps
    1. Created a Logical Column in BMM Layer for a cube dimension
    2. Opened the LTS Properties --> Column Mapping and gave below formula
    Lookup(SPARSE "Function1"."E:\502125500".""."Function2.txt"."ALIAS", 'NA', "TNWD06613.genpitfi01.og.ge.com"."GOGREP".""."GOGREP"."Gen5,Functions" )
    where
    Function1"."E:\502125500".""."Function2.txt"."ALIAS" ---> Column from File (Created File DSN and imported in RPD)
    "TNWD06613.genpitfi01.og.ge.com"."GOGREP" ---> Essbase Cube
    3. Dragged the Column to Presentation for Subject Area on Cube
    When I try to create a report on this column I get below error.
    Error Codes: OPR4ONWY:U9IM8TAC:OI2DL65P
    State: HY000. Code: 10058. [NQODBC] [SQL_STATE: HY000] [nQSError: 10058] A general error has occurred. [nQSError: 43113] Message returned from OBIS. [nQSError: 16001] ODBC error state: 22005 code: -3030 message: [Microsoft][ODBC Text Driver] Expression too complex. [nQSError: 16015] SQL statement execution failed. (HY000)
    SQL Issued: SELECT 0 s_0, "Essbase Test"."Functions"."Alias" s_1, "Essbase Test"."Functions"."Gen5,Functions" s_2, SORTKEY("Essbase Test"."Functions"."Gen5,Functions") s_3 FROM "Essbase Test" FETCH FIRST 65001 ROWS ONLY
    Can anybody help me on the same?

    You can script schemas from all DBs and in case of conflicts you will have to decide what to do.
    For example if you have the same SP or UDF on two DBs that does different things, you need to decide what to do with them.
    Only if the conflict is tables with the same structure and without KEY constraints and you can also script data and go on.
    If you have PK-FK relationships (which I bet is the case) you really need to design your changes based on them and I suspect you will end up scripting instead of putting boxes on SSIS.
    It really is a migration job for an experienced DBA and am afraid SSIS can do that much as
    Integrate.. not Migrate.

  • Text file storing in a database table

    I have a text file, that contains so many lines in it,..
    Now the problem is that.. I want to store my text file with the pl/sql code in a database table....
    plz suggest, how to create a table and how to store that text file in a database table.....

    The following code is tested and get the correct result (forms 10.1.2.0):
    Connected to:
    Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options
    SQL> create table tclob  (col_lob clob);
    Table created.
    Declare
      file  TEXT_IO.FILE_TYPE;
      text  varchar2(10000);
      L$Clob CLOB;
    Begin
      -- prepare CLOB object --
      dbms_lob.createtemporary( L$Clob, TRUE ) ;
      -- open text file --
      file := TEXT_IO.FOPEN( 'c:\forms.css', 'R' ) ;
      If TEXT_IO.IS_OPEN( file) Then
        -- start reazding file --
         Begin
           Loop
             TEXT_IO.GET_LINE( file, text );
             -- add the line to the CLOB variable --
                dbms_lob.writeappend (L$Clob, length(text), text);
           End loop;
        Exception
          when no_data_found then
            -- end of text file --
            TEXT_IO.FCLOSE(file);
          when others then
           null;
        End ;
        -- store the CLOB in the database --
        --Update tclob Set col_lob = L$Clob ;
        insert into tclob (col_lob) values (L$Clob);
        forms_ddl('commit');
           dbms_lob.freetemporary( L$Clob ) ;
      End if ;
    exception
         when others then
         message(error_code || '-' || error_text);
    End;Francois

  • Dropping log file in standby database

    Please,
    I need a help for the following issue:
    I'm making a technical documentation on various event that occur on dataguard configuraation, right now I drop a redo log group file on primary database, and when I try to drop the equivalent log group file on standby database I got the following error:
    SQL> alter database drop logfile group 3;
    alter database drop logfile group 3
    ERROR at line 1:
    ORA-01156: recovery in progress may need access to files
    this is the current state of the redolog file on standby database.
    SQL> select group#,members,status from v$log;
    GROUP# MEMBERS STATUS
    1 3 CLEARING_CURRENT
    3 3 CLEARING
    2 3 CLEARING
    Eventhough I do the following command on standby I also get an error.
    SQL> ALTER DATABASE CLEAR LOGFILE GROUP 3;
    ALTER DATABASE CLEAR LOGFILE GROUP 3
    ERROR at line 1:
    ORA-01156: recovery in progress may need access to files
    Can someone tell me how to drop on dataguard configuration the redolog file on primary database and their corresponding on standby database
    I'm working on 10 release 2, on Windows
    Thanks you

    Oracle Dataguard Concept and administration release 2,ref b14239: is my source but it doesn't work when trying to drop stanby group or logile member.
    For example, if the primary database has 10 online redo log files and the standby
    database has 2, and then you switch over to the standby database so that it functions
    as the new primary database, the new primary database is forced to archive more
    frequently than the original primary database.
    Consequently, when you add or drop an online redo log file at the primary site, it is
    important that you synchronize the changes in the standby database by following
    these steps:
    1. If Redo Apply is running, you must cancel Redo Apply before you can change the
    log files.
    2. If the STANDBY_FILE_MANAGEMENT initialization parameter is set to AUTO,
    change the value to MANUAL.
    3. Add or drop an online redo log file:
    ■ To add an online redo log file, use a SQL statement such as this:
    SQL> ALTER DATABASE ADD LOGFILE '/disk1/oracle/oradata/payroll/prmy3.log'
    SIZE 100M;
    ■ To drop an online redo log file, use a SQL statement such as this:
    SQL> ALTER DATABASE DROP LOGFILE '/disk1/oracle/oradata/payroll/prmy3.log';
    4. Repeat the statement you used in Step 3 on each standby database.
    5. Restore the STANDBY_FILE_MANAGEMENT initialization parameter and the Redo Apply options to their original states.
    Thank

Maybe you are looking for

  • Gcc 4.7.2 fails to find/(or something else with) default library

    Hi! I use the gcc compiler (version 4.7.2) but when I try to compile using the following: g++ test.cpp I have verified that the code itself is perfectly valid because the following error does not occur when compiling on another machine (with the same

  • Modified Pick and pack screen

    1-I create a check box on the Pick & pack selection screen. I'm using the 'top' property to position my check box at the same horizontal level than the 'expand' button. But the check box appears much higher than I expected. When I check the 'top' pro

  • Dragging and Dropping eBooks in iTunes

    I have iTunes 11.0.1 on my MacBook 2,1 that I use to sync my Music to my iPad, third generation.  I am told that I can use iTunes to also sync non Apple Store purchased eBooks purchases to my iPad.  I would do this by just dragging /dropping the eboo

  • Having problem placing a picutre with script

    hi all im kinda new to this i have 1 pic i want to place in specific coordinate i opened up a new document A4 and i opened my script i have a problem with the x and y coordinate the numbers doesnt make sense what i did? i manualy checked for the x an

  • Having problems setting alarm time since daylight savings time change today

    Since the change to Daylight Savings time this morning, I am unable to set my alarm correctly.  I set it to 2:40 a.m. but the time on the alarm shows 3:40 a.m.  I have all the proper settings correctly done.  I have powered down the phone numerous ti