Selecting Max Value from Huge Table

Dear Proffessionals
I have a huge table (20,000,000+ records) with the following columns:
[Time], [User], [Value]
The values in [Value] column can recur for a single User at a Time e.g.
2015-01-01, Me, X
2015-01-01, Me, Y
2015-01-01, Me, X
2015-01-02, Me, Z
2015-01-02, Me, X
2015-01-02, Me, Z
For each day, and for every user I want to have the maximum recurring value :
2015-01-01, Me, X
2015-01-02, Me, Z
to be inserted into another table.
PS: I want the MOST optimized way of achieving this functionality, bcause I am expecting a growth on the raw table over time, so PERFORMANCE is of great consideration.
I would really appreciate it, if somebody can help me.
Regards

I can think of two techniques based on the data selecticity
1) using row number function
2) using cross apply operator
USE Northwind;
-- Solution 1
SELECT S.SupplierID, S.CompanyName, CA.ProductID, CA.UnitPrice
FROM dbo.Suppliers AS S
  CROSS APPLY
    (SELECT TOP (10) *
     FROM dbo.Products AS P
     WHERE P.SupplierID = S.SupplierID
     ORDER BY UnitPrice DESC, ProductID DESC) AS CA
ORDER BY S.SupplierID, CA.UnitPrice DESC, CA.ProductID DESC;
-- Solution 2
WITH C AS
  SELECT S.SupplierID, S.CompanyName, P.ProductID, P.UnitPrice,
    ROW_NUMBER() OVER(
      PARTITION BY P.SupplierID
      ORDER BY P.UnitPrice DESC, P.ProductID DESC) AS RowNum
  FROM dbo.Suppliers AS S
    JOIN dbo.Products AS P
      ON P.SupplierID = S.SupplierID
SELECT SupplierID, CompanyName, ProductID, UnitPrice
FROM C
WHERE RowNum <= 10
ORDER BY SupplierID, ProductID DESC, UnitPrice DESC;
Best Regards,Uri Dimant SQL Server MVP,
http://sqlblog.com/blogs/uri_dimant/
MS SQL optimization: MS SQL Development and Optimization
MS SQL Consulting:
Large scale of database and data cleansing
Remote DBA Services:
Improves MS SQL Database Performance
SQL Server Integration Services:
Business Intelligence

Similar Messages

  • Select max date from a table with multiple records

    I need help writing an SQL to select max date from a table with multiple records.
    Here's the scenario. There are multiple SA_IDs repeated with various EFFDT (dates). I want to retrieve the most recent effective date so that the SA_ID is unique. Looks simple, but I can't figure this out. Please help.
    SA_ID CHAR_TYPE_CD EFFDT CHAR_VAL
    0000651005 BASE 15-AUG-07 YES
    0000651005 BASE 13-NOV-09 NO
    0010973671 BASE 20-MAR-08 YES
    0010973671 BASE 18-JUN-10 NO

    Hi,
    Welcome to the forum!
    Whenever you have a question, post a little sample data in a form that people can use to re-create the problem and test their ideas.
    For example:
    CREATE TABLE     table_x
    (     sa_id          NUMBER (10)
    ,     char_type     VARCHAR2 (10)
    ,     effdt          DATE
    ,     char_val     VARCHAR2 (10)
    INSERT INTO table_x (sa_id,  char_type, effdt,                          char_val)
         VALUES     (0000651005, 'BASE',    TO_DATE ('15-AUG-2007', 'DD-MON-YYYY'), 'YES');
    INSERT INTO table_x (sa_id,  char_type, effdt,                          char_val)
         VALUES     (0000651005, 'BASE',    TO_DATE ('13-NOV-2009', 'DD-MON-YYYY'), 'NO');
    INSERT INTO table_x (sa_id,  char_type, effdt,                          char_val)
         VALUES     (0010973671, 'BASE',    TO_DATE ('20-MAR-2008', 'DD-MON-YYYY'), 'YES');
    INSERT INTO table_x (sa_id,  char_type, effdt,                          char_val)
         VALUES     (0010973671, 'BASE',    TO_DATE ('18-JUN-2010', 'DD-MON-YYYY'), 'NO');
    COMMIT;Also, post the results that you want from that data. I'm not certain, but I think you want these results:
    `    SA_ID LAST_EFFD
        651005 13-NOV-09
      10973671 18-JUN-10That is, the latest effdt for each distinct sa_id.
    Here's how to get those results:
    SELECT    sa_id
    ,         MAX (effdt)    AS last_effdt
    FROM      table_x
    GROUP BY  sa_id
    ;

  • Need of SQL query in selecting distinct values from two tables

    hi,
    I need a query for selecting distinct values from two tables with one condition.
    for eg:
    there are two tables a & b.
    in table a there are values like age,sex,name,empno and in table b valuses are such as age,salary,DOJ,empno.
    here what i need is with the help of empno as unique field,i need to select distinct values from two tables (ie) except age.
    can anybody please help me.
    Thanks in advance,
    Ratheesh

    Not sure what you mean either, but perhaps this will start a dialog:
    SELECT DISTINCT a.empno,
                    a.name,
                    a.sex,
                    b.salary,
                    b.doj
    FROM    a,
            b
    WHERE   a.empno = b.empno;Greg

  • Min and Max values from entire table

     Hi,
     i have requirement in which i need to find the min and max values from the entire table.
    See the sample data 
    create table test
    Sal1 int,
    Sal2 int,
    Sal3 int
    insert into test values (100,700,5700)
    insert into test values (200,3300,5300)
    insert into test values (4400,1200,3500)
    insert into test values (5400,5600,3100)
    i want the output as 100 and 5700.. how can i achieve this in a single query. Please through some light on this topic..!
    Thanking you in advance
    Regards,
    Balaji Prasad B
    Balaji - BI Developer

    Below is an example with a subquery for each of the queries Mohammad posted in order to return both min and max in a single result set.
    SELECT ( SELECT MAX(Maxx) AS Maxx
    FROM test UNPIVOT
    ( Maxx FOR E IN ( Sal1, Sal2, Sal3 ) ) AS unpvt
    ) AS Maxx
    , ( SELECT MIN(Minn) Minn
    FROM test UNPIVOT
    ( Minn FOR E IN ( Sal1, Sal2, Sal3 ) ) AS unpvt
    ) AS Minxx;
    Dan Guzman, SQL Server MVP, http://www.dbdelta.com

  • How to select distinct values from a table when it has composite primary ke

    Hi
    I have the requirement like , I need to select distinct one column values from the table which has composite primary key. How to acheive this functioinality using view object.
    Eg : Table 1 has col1 and col2, col3
    col1 col2 col3
    1 A NA
    1 B NA
    2 A NA
    3 C NA
    2 D NA
    primary key (col1,col2)
    I have to select distinct col1.
    Thanks

    Hi
    I got the solution for above. By Creating the read only view object we can acheive this.
    thanks

  • Selecting a MAX value from a table & displaying it on a form using OCI PHP

    WinXP Pro SP3- 32bit
    Apache/2.2.22 (Win32) mod_fcgid/2.3.6 PHP/5.4.0
    PHP Version 5.4.0
    Oracle epxress 11g
    problem: output on form displaying Resource id #4 instead of the number from the table.
    (maybe it's number to string conversion issue?)
    $conn = oci_connect('system', 'mypassword', 'localhost/xe');
    $query = "SELECT MAX(CustNo)+1 AS MAXNUM FROM customer";
    $stmt = OCIParse($conn, $query);
    OCIExecute($stmt);
    OCIFetch($stmt);
    echo OCIResult($stmt, "MAXNUM")." will be next number.";
    //OCIFreeStatement($stmt); //frees the resources- use a the end!
    $daNextNo = $stmt;
    echo $daNextNo;
    //$daNextNo = OCIBindByName($stmt, ":text", &$form_text, -1); //not sure how this works if it'll fix the issue
    ?>
    <form name="Addcust" action="process_cust.php" method="post">
    <input type="text" name="CustNo" value="<?php echo $daNextNo;?>" /></br>
    output of complete code:
    45 will be next number.Resource id #4
    And inside the textbox it says: 45 will be next number.Resource id #4
    how can I put the 45 into $php variable so i can use it in a textbox or on a label?
    Thank you in advance .

    THanks. i couldn;t find the PHP section.
    I figured it out
    i used nextval() to convert the object to a string.
    <?php
    $conn = oci_connect('system', '1234', 'localhost/xe');
    if (!$conn) {
    $m = oci_error();
    trigger_error('Could not connect to database: '. $m['message'], E_USER_ERROR);
    $query = "SELECT MAX(CustNo)+1 AS MAXNUM FROM customer";
    //$daNextNo = $query;
    $stmt = OCIParse($conn, $query);
    OCIExecute($stmt);
    OCIFetch($stmt);
    echo OCIResult($stmt, "MAXNUM")." will be next number.";
    //OCIFreeStatement($stmt); //frees the resources- use a the end!
    $daNextNo = strval(OCIResult($stmt, "MAXNUM")." .");
    $daNextNo = strval(OCIResult($stmt, "MAXNUM"));
    echo "danextNo: ";
    echo $daNextNo;
    ?>
    <form name="Addcust" action="process_cust.php" method="post">
                                       <div>
                                       <dl>
                                            <dt>* <?php //echo $this->lang->line('cust_fn'); ?>: </dt>
                                            <dd><input type="text" name="CustNo" value="<?php echo $daNextNo;?>" /></dd>

  • To Select Max Value in the table

    Hello All,
    I want to select a record from a Z table which has the following fields.
    POSID
    STLNR
    DATUM
    UZEIT
    UNAME
    TCODE
    I have to select LAstest record depending upon the datum & uzeit field
    I am using like this.
        SELECT SINGLE UNAME
                      MAX (DATUM)
                      MAX (UZEIT) FROM Z48C_ZDIS_MD51 INTO
                      (G_R_MD51-UNAME,G_R_MD51-DATUM,G_R_MD51-UZETIT)
                                            WHERE POSID = WA_PRPS-POSID
                                            GROUP BY DATUM UZEIT.
    But Its giving syntax error.

    Select All records into internal table. Sort descending by date and time.  Read the first record.
    data: itab type table of ztable with header line.
    select * into itab from ztable.
    sort itab descending by datum uzeit.
    read table itab index 1.
    Regards,
    Rich Heilman

  • Cannot select different values from a table in a view object

    Hi everyone,
    I have two tables, Employee : ID, NAME, SALARY, NO_ROLE and role : ROLE_ID, ROLE_EMP. I join them with NO_ROLE and ROLE_ID. I created a view object with these two tables in JDeveloper, I made a LOV with ROLE_EMP and I want to be ale to create an employee with his ID, NAME, SALARY and by choosing a value in the ROLE_EMP list. The problem is that when I choose a ROLE_EMP, I cannot commit because the link between ROLE_EMP and ROLE_ID is not made correctly in ADF.
    In SQL, that's what I'm trying to do: INSERT INTO Employee VALUES (ID, NAME, SALARY, NO_ROLE) WHERE NO_ROLE = ROLE_ID of the ROLE_EMP I have selected.
    Thanks for your help ;)
    PS: I followed this tutorial to do this, but it is not exactly what I want to do: http://docs.oracle.com/cd/E18941_01/tutorials/jdtut_11r2_55/jdtut_11r2_55_2.html

    To be more generic, when I create an employee and select his role_emp (the name of the role) I want that the role_id of the role_emp to commit.

  • How to make validation in Bean and select value from another table

    I want to know how to select data from table in backing bean according to primary key i have
    the problem is that
    i have a table Employee_Salary contains Employee ids and their salary
    Empoloyee_Salary table
         Employee_ID      Number
         Employee_salary Number
    And Another table Called Employees
    Employees table
         Employee_ID     Number
         IsManager Varchar2 its value is [*Yes or NO*]
    and other columns that i don't care about this table
    i have on a jsff page an <af:table> this table is editable this is the Empoloyee_Salary table
    *i want to check before save or after insert if this employee is Manager [from Employees tabke(yes or no)] the salary*
    cannot be less that 100
    i want to know how to make this how to select the value from employees table according to the id i have in the employee_salary table how to make this and make this validation
    i have to select IsManager from Employees Table to see if this manager or no
    i want to know how to make this in a bean
    i use jdeveloper 11g
    and my project is ADF Fusion project
    and the page that have the Emplpyee_Salary table is JSFF
    thanks in advance

    You might want to write this code in a validator on the entity object if it should apply from every screen.
    If you want to access view objects from a backing bean the basics are here: http://download.oracle.com/docs/cd/E15523_01/web.1111/b31974/bcservices.htm#sthref918

  • How to pick max value from a column of a table using cursor and iteration

    Hello Everybody
    I have a table loan_detail
    and a column in it loan_amount
    now i want to pick values from this table using cursor and then by using iteration i want to pick max value from it using that cursor
    here is my table
    LOAN_AMOUNT
    100
    200
    300
    400
    500
    5600
    700i was able to do it using simple loop concepts but when i was trying to do this by using cursor i was not able to do it .
    Regards
    Peeyush

    SQL> SELECT MAX(sal) Highest_Sal,MIN(sal) Lowest_Sal FROM emp;
    HIGHEST_SAL LOWEST_SAL
           5000        800
    SQL> set serverout on
    SQL> DECLARE
      2    TYPE tmp_tbl IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;
      3    sal_tbl tmp_tbl;
      4    CURSOR emp_sal IS
      5      SELECT sal FROM emp;
      6    counter INTEGER := 1;
      7  BEGIN
      8    FOR i IN emp_sal LOOP
      9      sal_tbl(i.sal) := counter;
    10      counter := counter + 1;
    11    END LOOP;
    12    DBMS_OUTPUT.put_line('Lowest SAL:' || sal_tbl.FIRST);
    13    DBMS_OUTPUT.put_line('Highest SAL:' || sal_tbl.LAST);
    14  END;
    15  /
    Lowest SAL:800
    Highest SAL:5000
    PL/SQL procedure successfully completed.
    SQL> Even smaller
    SQL> DECLARE
      2    TYPE tmp_tbl IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;
      3    sal_tbl tmp_tbl;
      4    CURSOR emp_sal IS
      5      SELECT sal FROM emp;
      6    counter INTEGER := 1;
      7  BEGIN
      8    FOR i IN emp_sal LOOP
      9      sal_tbl(i.sal) := 1;
    10    END LOOP;
    11    DBMS_OUTPUT.put_line('Lowest SAL:' || sal_tbl.FIRST);
    12    DBMS_OUTPUT.put_line('Highest SAL:' || sal_tbl.LAST);
    13  END;
    14  /
    Lowest SAL:800
    Highest SAL:5000
    PL/SQL procedure successfully completed.
    SQL> Edited by: Saubhik on Jan 5, 2011 4:41 PM

  • Select MAX(BUDAT) from ztab (custom table) where... NOT work!

    We use the following statement where BUDAT is one of the fields in our custom table ztab:
    Select MAX(BUDAT) from ztab (custom table) where...
    When activating the above code, get the following error:
    "Unknown column name "MAX(BUDAT)". not determined until runtime, you cannot specify a field list."
    How to resolve this problem to get a max value of the field BUDAT in custom table ztab (it's not an internal table)?
    <REMOVED BY MODERATOR>
    Edited by: Alvaro Tejada Galindo on Apr 10, 2008 3:56 PM

    HI,
    Tyr having a space after and before BUDAT.
    ( BUDAT ).
    Hope it helps,
    Shreekant

  • Get selected row values from Table view control

    Hi ,
    I am using transaction ME23N, would like to access row values from item table for selected row. I have written a script as in screen shot and its giving me error at java script step two. I want to get the PR number from item table for selected row.
    With Regards
    Vishal Lokapur

    H Vishal,
    Can you please share how you were able to resolve the issue regarding the selected row
    in case of a table control .
    Regards

  • Copy selected values from a table control into another table control

    hi there,
    as seen in the subject i need to copy selected values from a table control into another table control in the same screen. as i dont know much about table controls i made 2 table controls with the wizard and started to change the code... right now im totally messed up. nothing works anymore and i don't know where to start over.
    i looked up the forums and google, but there is nothing to help me with this problem (or i suck in searching the internet for solutions)
    i have 2 buttons. one to push the selected data from the top table control into the bottom tc and the other button is to push selected data from the bottom tc into the top tc. does somebody has a sample code to do this?

    you're funny
    i still don't get it... can't believe, there is no tutorial or sample code around how to copy multiple selected rows from a tc.
    here's my code, maybe you can tell me exactly were i have to change it:
    tc1 = upper table control
    tc2 = lower table control
    SCREEN 0100:
    PROCESS BEFORE OUTPUT.
      MODULE status_0100.
      MODULE get_nfo. --> gets data from the dictionary table
      MODULE tc1_change_tc_attr.
      LOOP AT   it_roles_tc1
           INTO wa_roles_tc1
           WITH CONTROL tc1
           CURSOR tc1-current_line.
      ENDLOOP.
      MODULE tc2_change_tc_attr.
      LOOP AT   it_roles_tc2
           INTO wa_roles_tc2l
           WITH CONTROL tc2
           CURSOR tc2-current_line.
      ENDLOOP.
    PROCESS AFTER INPUT.
      LOOP AT it_roles_tc1.
        CHAIN.
          FIELD wa_roles_tc1-agr_name.
          FIELD wa_roles_tc1-text.
        ENDCHAIN.
        FIELD wa_roles_tc1-mark
          MODULE tc1_mark ON REQUEST.
      ENDLOOP.
      LOOP AT it_roles_tc2.
        CHAIN.
          FIELD wa_roles_tc2-agr_name.
          FIELD wa_roles_tc2-text.
        ENDCHAIN.
        FIELD wa_roles_tc2-mark
          MODULE tc2_mark ON REQUEST.
      ENDLOOP.
      MODULE ok_code.
      MODULE user_command_0100.
    INCLUDE PAI:
    MODULE tc1_mark INPUT.
      IF tc1-line_sel_mode = 2
      AND wa_roles_tc1-mark = 'X'.
        LOOP AT it_roles_tc1 INTO g_tc1_wa2
          WHERE mark = 'X'.    -
    > big problem here is, that no entry has an 'X' there
          g_tc1_wa2-mark = ''.
          MODIFY it_roles_tc1
            FROM g_tc1_wa2
            TRANSPORTING mark.
        ENDLOOP.
      ENDIF.
      MODIFY it_roles_tc1
        FROM wa_roles_tc1
        INDEX tc1-current_line
        TRANSPORTING mark.
    ENDMODULE.                    "TC1_MARK INPUT
    MODULE tc2_mark INPUT.
      IF tc2-line_sel_mode = 2
      AND wa_roles_tc2-mark = 'X'.
        LOOP AT it_roles_tc2 INTO g_tc2_wa2
          WHERE mark = 'X'.             -
    > same here, it doesn't gets any data
          g_tc2_wa2-mark = ''.
          MODIFY it_roles_tc2
            FROM g_tc2_wa2
            TRANSPORTING mark.
        ENDLOOP.
      ENDIF.
      MODIFY it_roles_tc2
        FROM wa_roles_tc2
        INDEX tc2-current_line
        TRANSPORTING mark.
    ENDMODULE. 
    thx for anybody who can help with this!

  • How to update value stored in a table when we select a value from a list

    In my jsp page i have given
    <html:select property="function" >
         <html:option value="select">Select Actions</html:option>
    <html:option value="open">Marks as open</html:option>
    <html:option value="closed">Marks as closed</html:option>
    </html:select>
    I am using lookup dispatch action in action class
    public class LoginAction extends LookupDispatchAction{
    protected Map getKeyMethodMap() {
              Map map=new HashMap ();
              map.put("open","open");
              map.put("closed","closed");
              return map;
    public ActionForward open(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response) throws Exception {
              System.out.println("success");
              return mapping.findForward("success");
         public ActionForward closed(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response) throws Exception {
              System.out.println("success");
              return mapping.findForward("success");
    but when i select a value from the list,the corresponding action is not occuring.What should i put in my code??

    Put your code between the code tags, it makes it so much easier to read.
    To do that, just highlight your code and click on the "code" found between " and the circled ?.
    From what I can read of your code I don't even see a table or a list, so i have no idea what you are trying to do. You probably need to show more code.
    JSG

  • Fetching more than one row from a table after selecting one value from the dropdown

    Hi Experts,
    How can we fetch more than one row from a table after selecting one value from the dropdown.
    The scenario is that I have some entries in the dropdown like below
      A               B               C        
    11256          VID          911256  
    11256          VID          811256
    11256          SONY      11256
    The 'B' values are there in the dropdown. I have removed the duplicate entries from the dropdown so now the dropdownlist has only two values.for eg- 'VID' and'SONY'. So now, after selecting 'VID' from the dropdown I should get all the 'C' values. After this the "C' values are to be passed to other methods to fetch some data from other tables.
    Request your help on this.
    Thanks,
    Preeetam Narkhede.

    Hi Preetam!
    I hope I understand your request proberly, since this is more about Java and less about WebDynpro, but if I'm wrong, just follow up on this.
    Supposed you have some collection of your original table data stored in variable "origin". Populate a Hashtable using the values from column "B" (let's assume it's Strings) as keys and an ArrayList of whatever "C" is (let's assume String instances, too) as value (there's a lot of ways to iterate over whatever your datasource is, and since we do not know what your datasource is, maybe you'll have to follow another approach to get b and c vaues,but the principle should remain the same):
    // Declare a private variable for your Data at the appropriate place in your code
    private Hashtable temp = new Hashtable<String, ArrayList<String>>();
    // Then, in the method you use to retrieve backend data and populate the dropdown,
    // populate the Hashtable, too
    Iterator<TableData> a = origin.iterator();
    while (a.hasNext()) {
         TableData current = a.next();
         String b = current.getB();
         String c = current.getC();
         ArrayList<String> values = this.temp.get(b);
         if (values == null) {
              values = new ArrayList<String>();
         values.add(c);
         this.temp.put(b, values);
    So after this, you'll have a Hashtable with the B values als keys and collections of C values of this particular B as value:
    VID --> (911256, 811256)
    SONY --> (11256)
    Use
    temp.keySet()
    to populate your dropdown.
    After the user selects an entry from the dropdown (let's say stored in variable selectedB), you will be able to retrieve the collection of c's from your Hashtable
    // In the metod you handle the selection event with, get the c value collection
    //and use it to select from your other table
    ArrayList<String> selectedCs = this.temp.get(selectedB);
    // now iterate over the selectedCs items and use each of these
    //to continue retrieving whatever data you need...
    for (String oneC : selectedCs) {
         // Select Data from backend using oneC in the where-Clause or whatever...
    Hope that helps
    Michael

Maybe you are looking for