Optimising Postgres performance in Linux VMs

What is the optimal deployment for high-performance Postgres databases on an Ubuntu Linux VM - beyond standard A-series and D-series basic configurations?
This will be useful to many of our research users particularly.
Thanks,
 Kenji Takeda (Microsoft Research)
Kenji Microsoft Research Connections http://research.microsoft.com/collaboration/

Hi,
On these new D-series sizes, the temporary drive (/mnt or
/mnt/resource on Linux) are local SSDs. This high-speed local disk is best used for workloads that replicate across multiple instances, like MongoDB, or can leverage this high I/O disk for a local and temporary
cache. (Note, these drives are not guaranteed to be persistent. Thus, while physical hardware failure is rare, when it occurs, the data on this disk may be lost, unlike your OS disk and any attached durable disks that are persisted in Azure Storage.) The D-series sizes
offer an optimal configuration for running workloads that require increased processing power and faster local disk I/O than A-series sizes.
In addition, you can group two or more virtual machines in an Availability Set to provide redundancy as it ensures that during either a planned or unplanned maintenance event, at least one virtual machine will be available and meet the 99.95% Azure
SLA.
Best regards,
Susie
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact [email protected]

Similar Messages

  • Need help in optimising the performance of a query

    Need help in optimising the performance of a query. Below is the query that is executed on TABLE_A, TABLE_B and TABLE_C with record counts as 10M, 10m and 42 (only) respectively and it takes around 5-7 minutes to get 40 records:
    SELECT DISTINCT a.T_ID_, a.FIRSTNAME, b.T_CODE, b.PRODUCT,
    CASE WHEN TRUNC(b.DATE) +90 = TRUNC(SYSDATE) THEN -90 WHEN TRUNC(b.DATE) +30 = TRUNC(SYSDATE) THEN -30 ELSE 0 END AS T_DATE FROM TABLE_B b
    INNER JOIN TABLE_A a ON (a.T_ID_ = b.T_ID_) LEFT JOIN TABLE_C c ON b.PRODUCT = c.PRODUCT
    WHERE b.STATUS = 'T' AND (b.TYPE = 'ACTION'
    AND ( TRUNC(b.DATE) + 1 = TRUNC(SYSDATE) ) ) AND b.PRODUCT = 2;
    Note: Indices on the join columns are available in the respective tables
    Please let me know if there is any better way to write it.
    Edited by: 862944 on Aug 18, 2011 9:52 AM

    862944 wrote:
    Need help in optimising the performance of a query. Below is the query that is executed on TABLE_A, TABLE_B and TABLE_C with record counts as 10M, 10m and 42 (only) respectively and it takes around 5-7 minutes to get 40 records:
    SELECT DISTINCT a.T_ID_, a.FIRSTNAME, b.T_CODE, b.PRODUCT,
    CASE WHEN TRUNC(b.DATE) +90 = TRUNC(SYSDATE) THEN -90 WHEN TRUNC(b.DATE) +30 = TRUNC(SYSDATE) THEN -30 ELSE 0 END AS T_DATE FROM TABLE_B b
    INNER JOIN TABLE_A a ON (a.T_ID_ = b.T_ID_) LEFT JOIN TABLE_C c ON b.PRODUCT = c.PRODUCT
    WHERE b.STATUS = 'T' AND (b.TYPE = 'ACTION'
    AND ( TRUNC(b.DATE) + 1 = TRUNC(SYSDATE) ) ) AND b.PRODUCT = 2;
    Note: Indices on the join columns are available in the respective tables
    Please let me know if there is any better way to write it.
    Edited by: 862944 on Aug 18, 2011 9:52 AM[When Your Query Takes Too Long|https://forums.oracle.com/forums/thread.jspa?messageID=1812597]

  • Setting IPs in linux VMs

    Hello.
    I'm having the following problem.I want to set the IPs of a linux(centos 6.4) vm but i can't.
    I have  a)installed the 3.4 integration services b)used kernel 3.12.5 with hyper-v support.
    In both cases i always get in dmesg
    hv_utils: KVP: IP injection functionality not available
    hv_utils: KVP: Upgrade the KVP daemon
    hv_utils: KVP: user-mode registering done.
    Moreover i use the following in order to read :
    $VM = Get-WmiObject -Namespace 'root\virtualization\v2' -Class 'Msvm_ComputerSystem'  | Where-Object { $_.ElementName -eq "FooBar" }
    $VMSettings = $vm.GetRelated('Msvm_VirtualSystemSettingData') | Where-Object { $_.VirtualSystemType -eq 'Microsoft:Hyper-V:System:Realized' }    
    $VMNetAdapters = $VMSettings.GetRelated('Msvm_SyntheticEthernetPortSettingData')
    $VMNetAdapters
    I have also tried  $VMNetAdapters = $VMSettings.GetRelated('Msvm_EmulatedEthernetPortSettingData')  using a legacy card.
    The previous code was taken from http://www.ravichaganti.com/blog/?p=2766
    In a few words i would like to assign IPs in linux VMs.
    Thank you in advance!
    ps:I have now seen that 3.5 version of integration services has been released yesterday and it says that it supports static ip injection.So the previous versions didn't support ip injection at all?

    Hi tasoss,
    I did a little bit of research and unfortunately there are only two options for you:
    a) Either upgrade to CentOS 6.5
    or
    b) Apply the fix yourself.
    The following thread further attests to the above suggestions:
    https://www.centos.org/forums/viewtopic.php?f=13&t=44118
    If you'd like to compile the fix yourself then you should backport the following commits:
    1) Drivers: hv: util: Correctly support ws2008R2 and earlier
    http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/hv/hv_kvp.c?id=3a4916050ba2e0f1d114ef540abdf02b2b173e61
    2) Drivers: hv: remove HV_DRV_VERSION
    http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/hv/hv_kvp.c?id=cfc25993e81f3fa68481d062be634d33184d5eae
    3) Drivers: hv: util: Fix a bug in version negotiation code for util services
    http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/hv/hv_kvp.c?id=6741335bc7294548ac95cb1f4671991ff30da193
    Please let me know if you need more information.
    Thanks,
    Abhishek Gupta, Program Manager, Linux Integration Services

  • Best methods to speed up and increase performance of Linux?

    I would like this thread to be dedicated to various speed up techniques and performance tweaks for Linux and especially arch.
    K.Mandla offers quite a bit of interesting tweaks in the guide "Howto: Set up Hardy for speed", most of it applies to all linux distros.
    http://kmandla.wordpress.com/2008/05/04 … for-speed/
    If you have come across any interesting information relating to getting faster and better performance from Linux please do not hesitate to post. Thanks.

    PrimoTurbo wrote:
    I would like this thread to be dedicated to various speed up techniques and performance tweaks for Linux and especially arch.
    K.Mandla offers quite a bit of interesting tweaks in the guide "Howto: Set up Hardy for speed", most of it applies to all linux distros.
    http://kmandla.wordpress.com/2008/05/04 … for-speed/
    If you have come across any interesting information relating to getting faster and better performance from Linux please do not hesitate to post. Thanks.
    You will most likely end up wasting a lot of time for very little gain.
    There are no miracle solutions, you either use faster apps or buy faster hardware.
    The tweaks which are truly interesting without major drawbacks usually make their way on a default system, one way or another.
    For example, for directory index on ext3 :
    http://wiki.archlinux.org/index.php/Ext … ystem_Tips : "Note: Directory indexing is activated by default in Archlinux via /etc/mke2fs.conf"
    So if you want an advice, there are areas more interesting to explore, like programming for instance. You might end up having the capability to really improve apps performance, or write your own lighter alternatives.

  • What are the advantages of using Hyper-V's synthetic drivers with Linux VMs?

    We are moving our virtualization platform from Citrix's XenServer to Hyper-V on Windows Server 2008 R2. As part of this project I need to migrate, in one form or another, some Debian Linux servers over to Hyper-V. I have successfully built a Debian-based
    server on our new Hyper-V platform and I'm beginning to test it.
    Debian 6 (Squeeze) uses the 2.6.32 kernel which includes the Hyper-V synthetic drivers, but it is not considered a supported operating guest operating system by Microsoft. I'm a little hesitant in trying to use them unless there's a compelling reason to,
    as other folks have had trouble
    What advantages do the Hyper-V synthetic drivers offer over the emulated drivers?
    For those of you who have experience with the Xen hypervisor, is using the synthetic drivers analgous to para-virtualizing a guest operating system?
    Are then any noteworthy dangers or drawbacks of NOT using the synthetic drivers?
    Why should I bother to either a) deal with the reported instability of the Hyper-V drivers currently in the kernel, b) try to build a newer kernel, or c) try to make the Virtual Machine Additions work with a distribution
    they weren't designed for when everything seems to "just work"?

    The synthetic drivers 'talk' more directly to the actual hardware, bypassing most of the hypervisor (for common data operations). This dramatically cuts down on the hypervisor overhead related to most network activity.
    If your server doesn't communicate much on the network, or if your hardware is well undercommitted, you should be fine with the emulated drivers. There's definitely a performance penalty for doing this however.

  • Solaris 10 NFS performance on Linux running ws3 update 3

    Hope someone can help me sort out this problem.
    Dear Support.
    We are having a Solaris/Sparc file server running Solaris 10. The Solaris machine act as a NFS file server. We encounter very poor NFS performance when copying files to and from a filesystem via Linux NFS
    I have set up a very simple test scenario. Created a tar file, size around 3 GB. The file is sitting on a SAN system. The file it self is created on a Solaris 10 UFS filesystem.
    Solaris E240 Solaris 8 NFS, GB interface copy to and from the same disk via NFS
    timecp /seis/seis600_new/usr.tar /seis/seis600_new/new1.tar
    real 2m18.91s
    user 0m0.11s
    sys 0m29.72s
    IBM/AMD 64 bit Linux WS3 U5, GB interface NFS to and from the same disk via NFS.
    time cp /seis/seis600_new/usr.tar /seis/seis600_new/new1.tar
    real 6m24.670s
    user 0m0.130s
    sys 0m21.860s
    Also run the test on other Linux boxes with similar results?
    The funny part of this is that I can reproduce the performance problem on other SUN systems among the SUN Blade 2000 with 8 GB Ram
    Let me wrap up.
    Always bad NFS performance when using NFS between Solaris NFS server and Linux client.
    Not always bad performance when using NFS between Solaris server and clients.

    It's been a while since I was doing linux->solaris nfs, so bear with me as I clear out the cobwebs.
    First things to check: Mount options for the nfs mount to the server.
    Which versions of nfs are you using?(v2, v3?) Solaris uses version 3 mounts by default.
    What's your wsize and rsize for reads and writes?
    I believe linux is limited to using 8k r/w block sizes. Solaris will let you use r/wsize up to 32k in nfsv3, which would really help with larger data transfers.
    nfsv3 has a number of performance enhancements over v2, so give that a shot with a larger block size.
    nfsvers=3,wsize=8192,rsize=8192,nolock,intr
    Also experiment with your locking options, that might help some.
    NFS has always been a fairly weak point for linux.
    A few links for reference:
    http://www.scd.ucar.edu/hps/TECH/LINUX/linux.html
    http://nfs.sourceforge.net/
    Cheers && good luck,
    fptt.

  • Swing thread performance in linux

    Hello,
    I am writting very small swing application for learning students...
    There is Robot object(s) which can walk through Panel and draw lines...
    But I have there some performance problems with threads in linux...
    In Windows is everything ok... but not in linux...
    code is very short... every Robot object has reference to PaintPanel
    extends JPanel. When robot do forward method, it paints DrawableLine.
    Painting is following through PaintPanel.drawShape(final Drawable shape)
    In MainFrame is little example method runRobots() which draw some
    pictures. When I added this method (runRobors) to thread, it is going
    very slow on linux...
    sources are here:
    http://homer.dactos.com/~hlavki/robot.zip
    thanks, miso

    camickr,
    Good call. that was what it wanted. Runs like a scalded dog without any flicker at this point.
    Wrapped the calls that would result in adding a ThreadMonitor, Removing a ThreadMonitor or updating a ThreadMonitor with
    SwingUtilities.invokeLater(new Runnable(){
        public void run(){
            // do something here
    }) ;and the ui is very well mannered now.
    There is one gotcha for anyone who comes along after and reads this thread. This change increases the app to use more heap space and at full throttle I was able to bring the application to its knees.
    Just something to be aware of.
    camickr, once again thank you.
    PS.

  • Slow performance on Linux for visual effects

    Visual effects such as Transitions perform very poor on linux "wheezy" compared to windows. I noticed this on different PC's, checked for Java7 and Java8.
    If the UI contains many objects then the transition sometimes does not even appear.
    I do not think this is graphic card related since videos play quite ok.
    I use the default ATI driver without Xorg.conf file and  installed the xcompmgr
    and tried several options, such as
    Option "Composite" "Enable"
    or
    Option        "backingstore" "true"   
    Option        "AllowGLXWithComposite" "true"
    This did not speed up things, are there other things that I could do to improve the performance ?

    Oracle JDK 8 and JRE 8 Certified System Configurations
    On Linux platforms, graphic hardware acceleration is only supported for Nvidia cards (proprietary drivers only).
    I don't know if that is the cause of your performance issues, but is probably something to consider.

  • The Azure team has released the Azure Diagnostics extension for Linux VMs

    You can now ingest Syslog data into OMS from Azure Linux VMsThe Azure team has released the Azure Diagnostics extension for Linux VMsIf you use Azure Diagnostics, Operations Management Suite can ingest syslog data from storage accounts configured to store your diagnostics data. As of the publication of this blog post, OMS can ingest all logs routed through syslog or RsyslogD.
    To enable this functionality, follow the steps below:1. Enable Azure Diagnostics for your Linux machines2. In the Classic Azure Portal find your Operational Insights workspace.Note: If you can’t find your OMS workspace here, it means you haven’t linked your workspace to your Azure Subscription – follow the steps to link workspaces here
    This topic first appeared in the Spiceworks Community

    The scripts you need for Linux will be posted on OTN soon.

  • RAC 10g on Linux VMs problems with smp

    I'm following Vincent Chan's article "Install Oracle RAC 10g on Oracle Enterprise Linux Using VMware Server" in order to get up to speed on Oracle RAC on Linux. I've had good success so far but it appears that when the kernel was created it was created as smp even though I only had processor count set to 1. I went on to choose the oracleasm-2.6.9-42.0.0.1EL driver for UP kernel not realizing I had an smp kernel. Now that I'm trying to create ASM disks I get the following message:
    Writing Oracle ASM library driver configuration: [ OK ]
    Loading module "oracleasm": Unable to load module "oracleasm"
    [FAILED]
    I'm assuming this problem is due to the wrong asm driver being installed.
    So my question is, since I truly have a single processor virtual machine, should I try to rebuild the kernel as UP? Or should I try to re-install the smp version of oracleasm and keep the kernal as smp with 1 processor?
    Additional info:
    # rpm -qa | grep asm
    ibmasm-3.0-7
    oracleasm-support-2.0.3-2
    oracleasmlib-2.0.2-1
    oracleasm-2.6.9-42.0.0.0.1.EL-2.0.3-2
    # uname -r
    2.6.9-42.0.0.0.1.ELsmp

    Hi Frits,
    I had the same problem but could not find the right rpm.
    Can you give the URL?
    My problem:
    [root@rac2 ~]# /etc/init.d/oracleasm configure
    Configuring the Oracle ASM library driver.
    This will configure the on-boot properties of the Oracle ASM library
    driver. The following questions will determine whether the driver is
    loaded on boot and what permissions it will have. The current values
    will be shown in brackets ('[]'). Hitting <ENTER> without typing an
    answer will keep that current value. Ctrl-C will abort.
    Default user to own the driver interface []: oracle
    Default group to own the driver interface []: dba
    Start Oracle ASM library driver on boot (y/n) [n]: y
    Fix permissions of Oracle ASM disks on boot (y/n) [y]: y
    Writing Oracle ASM library driver configuration: [  OK  ]
    Loading module "oracleasm": Unable to load module "oracleasm"
    [FAILED]
    [root@rac2 cmshare]# uname -r
    2.6.9-42.0.0.0.1.ELsmp
    [root@rac2 cmshare]# rpm -qa | grep oracleasm
    oracleasmlib-2.0.2-1
    oracleasm-support-2.0.3-2
    oracleasm-2.6.9-42.0.0.0.1.EL-2.0.3-2
    Regards,
    Chungmen

  • Performance (Linux or Solaris)

    Hi folks,
    We have been asked by many clients which performs better / faster (Linux vs Solaris).
    The environment is:
    Intel Xeon Server Hardware
    Local Disk (SAS) - RAID 5
    Single Server install
    Single Location
    VirtualBox as Hypervisor
    I am asking for your feedback / experience with Linux vs Solaris implementing VDI 3.4.1

    Hi,
    we have done some extensive testing on this. One of our clients had some severe issues with performance on Linux after a migration from solaris to linux (on the same hardware) the system load/utilization went up by a factor of 4-5 (server could easily handle 200 sessions with solaris and only 40-50 with linux). We never managed to find the root cause (or a solution) for this and migrated back to solaris.
    But we tried to replicate this issue in the lab (without success). So what we did was this:
    Install Solaris 10 and Linux on one hard drive each and also install SRS 5.3.1 (we also tried this with a few older versions). We set up kiosk sessions to connect to a windows terminal server (we did not bother to compare virtualbox perfoamce) and launch a web browser to a website with loads of flash and moving content. We had about 20 DTUs connect to this server and monitored the system load, utilization and so on of the Xnewt processes. At one point we also automatically started some audio stream to measure the performance of utaudiod. The result (for 2 different servers) was that linux and solaris are really close to each other in performance terms. Solaris showed a little less utilization, about 5-10% maybe. But really quite negligable.
    Long story short: we have seen really poor performance with linux once but in all other cases linux performed almost as well as solaris. And we never compared the virtualbox hypervisors which might be yet another story.
    Bjoern

  • Linux Performance Monitoring Scripts

    Hi Friends,
    My boss ask me to give weekly reports about the performance of the server.
    He wants the report in excel formant.
    I heard about vmstat, iostat,sar
    Are these the only command I need to monitor the performance of linux?
    Can you share me some scripts about monitoring linux performance that save data into txt file? and
    is being gathered (stats) every hour on top of the hour?
    Thanks a lot,
    MsK

    Thanks gentlemen,
    But I can not grasp either the ideas that you are explaining. :(
    It is not helpful that at some point or at all points, the report said that all the CPU and MEMORY were 100% used?
    At least there is a trigger or reason to start investigating was is running on the server.
    For example
    [root@oracleprod ~]# vmstat 5 10
    procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
    r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
    3  0 215040  21612  18548 8042292    3    4   406   129   12    13 14  1 82  3
    2  0 215040  25484  18480 8037940    0    0     2   192 1037   522 21  4 74  0
    1  0 215040  25476  18496 8037924    0    0     0    32 1013   443 21  5 75  0
    1  0 215040  25156  18512 8037908    0    0     0    58 1010   452 21  5 74  0
    1  0 215040  24772  18532 8038148    0    0    24    46 1008   482 21  5 74  0
    1  0 215040  24540  18548 8038132    0    0     0    45 1006   501 21  5 74  0
    1  0 215040  24540  18564 8038116    0    0     0    23 1006   439 22  3 75  0
    1  0 215040  24156  18580 8038100    0    0     0   302 1061   558 22  4 74  0
    2  0 215040  24028  18596 8038084    0    0     0    56 1014   449 24  1 74  0
    1  0 215040  23772  18604 8038076    0    0     2    22 1020   636 26  1 73  0Does vmstat not telling whats happening to system? or at these statistics useless :(
    Thanks a again

  • Performance of synchronized blocks on Linux much worse than on Windows

    Our application is a J2EE application currently deployed in WebLogic 6.1 which utilizes Oracle iFS as a content store. We access the iFS CMS through the api's provided in the cmdsk.jar. Our reference implementation has been Solaris and that's where most of our testing has been done so far. However, we recently began doing performance tests on Linux and ran into a strange problem. We found that we were not getting sufficient throughput and that this was coupled with underutilization of CPU resources on the Application Server (AS) node. With sufficient DB resources, which we had in this case, we expect to see full CPU utilization on the AS tier, so we knew something was wrong.
    A great deal of investigation revealed that all active threads were stuck at the same spot accessing an iFS cache. We found that this cache is implemented as a hashtable and accesses to hashtables are synchronized. However, we were not seeing the same problem on other platforms (Solaris and Windows) with dual CPU machines.
    In order to simplify things, we developed a test program to look at what we felt to be the root cause. This test program, which I will attach, essentially creates a hashtable then a number of threads (as specified by the user). These threads then use the hashtable.get method to access the hashtable concurrently and time how long it takes. The info is then returned at the end of the run.
    We ran a set of tests where we increased the number of threads from 1 to 100. This test was run once with Windows 2000 and once with Red Hat Linux 2.1 (kernel 2.4.9-e.12). The exact same hardware (dual 1.2 GHz PIII) was used in both cases. The results show that we were clearly able to reproduce the behaviour with this simple program. What these results show is that with one thread, access times are exactly the same for Windows and Linux. However, as we increase the number of threads, access times for Linux increase at a much greater rate than Windows. We also see underutilization of CPU resources on Linux while Windows uses all available.
    The first round of tests was run with Sun's JDK 1.3.1. We repeated the testing with 1.4.1 and found that there was no significant difference. I just tried it quickly with 1.4.2 and found the behaviour to be much the same. We also altered the program to use an unsynchronized hashmap instead of a hashtable and, as expected, performance on Linux was much better. CPU was fully utilized during the test.
    In order to determine if this was a problem "in the VM" or "in the kerenel", I tried repeating it with the JRocket VM provided by WebLogic. When running the same test with the JRocket VM, we saw performance roughly equivalent to what we saw with a hashmap and the Sun VM. That is, JRocket did not have the same problem as the Sun VM.
    So, it appears that there is something with the implementation of synchronized blocks in the Sun VM that is problematic, especially when compared to other platforms. It is likely that the problem lies in a combination of how the synchronization is implemention is based on the underlying operating system infrastructure. At this point, we don't really have a way to deal with the problem, aside from deploying multiple WebLogic nodes on the same 2 CPU physical AS node. Changes to the iFS infrastructure would help, but that's beyond our control. I've searched the bug database and have not seen mention of this problem. With the test program, it can be easily reproduced, but I wanted to run it by the folks on this list before submitting a bug.
    Thanks,
    Justin

    # File #1: Driver.java
    import java.io.*;
    import java.util.HashMap;
    import java.util.Hashtable;
    * Created on May 13, 2003
    * @author llai
    * Driver to test the access of hash table on various platforms.
    public class Driver
         public static void main(String[] args)
              * Obtain the following input values:
              * 1. Number of Elements
              *      - The number of elements in the hash table
              * 2. Size of Element
              * - The size of each element in the hash table
              * 3. Number of threads
              * - The number of concurrent threads in the test
              * 4. Focused Access
              * - Determines whether each thread access the same element or a random element
              * 5. Number of Accesses
              * - The number of accesses which each thread will complete before exiting
              * 6. Start offset
              * - The number of accesses at the beginning which will not be timed
              * 7. End offset
              * - The number of accesses at the end which will not be timed.
              * Initialize all input variables.
              * SET DEFAULT VALUES HERE...
              int numberOfElements = 10000;
              int sizeOfElement = 100;
              int numberOfThreads = 25;
              boolean focusedAccess = true;
              int numberOfAccesses = 1000000;
              int startOffset = 100000;
              int endOffset = 100000;
              boolean mapOrTable = false;
              * Input number of elements.
              try
                   BufferedReader in =
                        new BufferedReader(new InputStreamReader(System.in));
                   String tempInput;
                   boolean inputInvalid = true;
                   while (inputInvalid)
                        System.out.print(
                             "Enter a non-negative value for the number of elements to be in the hashtable: ");
                        tempInput = in.readLine();
                        if (tempInput.equals(""))
                             System.out.println("default value used.\n");
                             inputInvalid = false;
                        else
                             try
                                  numberOfElements = Integer.parseInt(tempInput);
                                  if (numberOfElements >= 0)
                                       inputInvalid = false;
                                  else
                                       System.out.println(
                                            "You entered a negative value for the number of elements. Try again.");
                             catch (NumberFormatException nfe)
                                  System.out.println(
                                       "You entered an invalid value for the number of elements. Try again.");
                   * Input size of elements.
                   inputInvalid = true;
                   while (inputInvalid)
                        System.out.print(
                             "Enter the size of each element in the hash table (ie. length of a string): ");
                        tempInput = in.readLine();
                        if (tempInput.equals(""))
                             System.out.println("default value used.\n");
                             inputInvalid = false;
                        else
                             try
                                  sizeOfElement = Integer.parseInt(tempInput);
                                  if (sizeOfElement >= 0)
                                       inputInvalid = false;
                                  else
                                       System.out.println(
                                            "You entered a negative value for the size of elements. Try again.");
                             catch (NumberFormatException nfe)
                                  System.out.println(
                                       "You entered an invalid value for the size of elements. Try again.");
                   * Input number of threads.
                   inputInvalid = true;
                   while (inputInvalid)
                        System.out.print(
                             "Enter the number of concurrent threads in the test: ");
                        tempInput = in.readLine();
                        if (tempInput.equals(""))
                             System.out.println("default value used.\n");
                             inputInvalid = false;
                        else
                             try
                                  numberOfThreads = Integer.parseInt(tempInput);
                                  if (numberOfThreads >= 0)
                                       inputInvalid = false;
                                  else
                                       System.out.println(
                                            "You entered a negative value for the number of threads. Try again.");
                             catch (NumberFormatException nfe)
                                  System.out.println(
                                       "You entered an invalid value for the number of threads. Try again.");
                   * Input focused access option.
                   inputInvalid = true;
                   while (inputInvalid)
                        System.out.print(
                             "Focused access? Determines whether each thread access the same element or a random element (y/n or blank for default): ");
                        String tempFocusedAccess = in.readLine();
                        if (tempFocusedAccess.equals("y"))
                             focusedAccess = true;
                             inputInvalid = false;
                        else if (tempFocusedAccess.equals("n"))
                             focusedAccess = false;
                             inputInvalid = false;
                        else if (tempFocusedAccess.equals(""))
                             System.out.println("default value used.\n");
                             inputInvalid = false;
                        else
                             System.out.println(
                                  "You entered an invalid input for focused access. Try again.");
                   * Input number of accesses.
                   inputInvalid = true;
                   while (inputInvalid)
                        System.out.print(
                             "Enter a non-negative value for the number of accesses each thread will complete: ");
                        tempInput = in.readLine();
                        if (tempInput.equals(""))
                             System.out.println("default value used.\n");
                             inputInvalid = false;
                        else
                             try
                                  numberOfAccesses = Integer.parseInt(tempInput);
                                  if (numberOfAccesses >= 0)
                                       inputInvalid = false;
                                  else
                                       System.out.println(
                                            "You entered a negative value for the number of accesses per thread. Try again.");
                             catch (NumberFormatException nfe)
                                  System.out.println(
                                       "You entered an invalid value for the number of accesses per thread. Try again.");
                   * Input start offset.
                   inputInvalid = true;
                   while (inputInvalid)
                        System.out.print("Enter the start offset: ");
                        tempInput = in.readLine();
                        if (tempInput.equals(""))
                             System.out.println("default value used.\n");
                             inputInvalid = false;
                        else
                             try
                                  startOffset = Integer.parseInt(tempInput);
                                  if (startOffset <= numberOfAccesses
                                       && startOffset >= 0)
                                       inputInvalid = false;
                                  else
                                       System.out.println(
                                            "You entered an invalid input for start offset. Try again.");
                             catch (NumberFormatException nfe)
                                  System.out.println(
                                       "You entered an invalid value for the start offset. Try again.");
                   * Input end offset.
                   inputInvalid = true;
                   while (inputInvalid)
                        System.out.print("Enter the end offset: ");
                        tempInput = in.readLine();
                        if (tempInput.equals(""))
                             System.out.println("default value used.\n");
                             inputInvalid = false;
                        else
                             try
                                  endOffset = Integer.parseInt(tempInput);
                                  if (endOffset <= numberOfAccesses && endOffset >= 0)
                                       inputInvalid = false;
                                  else
                                       System.out.println(
                                            "You entered an invalid input for end offset. Try again.");
                             catch (NumberFormatException nfe)
                                  System.out.println(
                                       "You entered an invalid value for the end offset. Try again.");
                   * Create a Hashtable, or a Hashmap?
                   inputInvalid = true;
                   while (inputInvalid)
                        System.out.print(
                             "Use HashMap or HashTable (enter m for HashMap, t for HashTable)? ");
                        String tempMapTable = in.readLine();
                        if (tempMapTable.equals("m"))
                             mapOrTable = true;
                             inputInvalid = false;
                        else if (tempMapTable.equals("t"))
                             mapOrTable = false;
                             inputInvalid = false;
                        else if (tempMapTable.equals(""))
                             System.out.println("default value used.\n");
                             inputInvalid = false;
                        else
                             System.out.println(
                                  "You entered an invalid input. Try again.");
              catch (IOException i)
                   System.out.println("IOException caught: " + i.getMessage());
              * End of obtaining input section. *****
              * Create a HashTable or HashMap of the size and distribution specified.
              Object hash;
              if (mapOrTable)
                   // create a HashMap
                   hash = new HashMap(numberOfElements);
              else
                   // create a HashTable
                   hash = new Hashtable(numberOfElements);
              char c[] = new char[sizeOfElement];
              for (int i = 0; i < numberOfElements; i++)
                   if (mapOrTable)
                        ((HashMap) hash).put(new Integer(i), new String(c));
                   else
                        ((Hashtable) hash).put(new Integer(i), new String(c));
              c = null; // Explicitly release c.
              * Create the specified number of threads.
              AccessorThread accessor[] = new AccessorThread[numberOfThreads];
              long seed;
              for (int i = 0; i < numberOfThreads; i++)
                   // Have a different seed for creating random number for each thread.
                   seed = System.currentTimeMillis() + 77 * i;
                   accessor[i] =
                        new AccessorThread(
                             hash,
                             numberOfElements,
                             focusedAccess,
                             numberOfAccesses,
                             startOffset,
                             endOffset,
                             seed);
                   // Start the thread.
                   accessor.start();
              * Make main wait for all threads to die before continuing.
              try
                   for (int i = 0; i < numberOfThreads; i++)
                        accessor[i].join();
              catch (InterruptedException ie)
                   System.out.println(
                        "InterruptedException caught: " + ie.getMessage());
              * Threads have now been run, so create a statistical report.
              System.out.println(
                   "The following settings were used for this test: \n"
                        + "Number of elements: "
                        + numberOfElements
                        + "\n"
                        + "Size of elements: "
                        + sizeOfElement
                        + "\n"
                        + "Number of threads: "
                        + numberOfThreads
                        + "\n"
                        + "Focused access: "
                        + focusedAccess
                        + "\n"
                        + "Number of accesses: "
                        + numberOfAccesses
                        + "\n"
                        + "Start offset: "
                        + startOffset
                        + "\n"
                        + "End offset: "
                        + endOffset);
              System.out.print("HashMap or HashTable: ");
              if (mapOrTable)
                   System.out.println("HashMap.\n");
              else
                   System.out.println("HashTable.\n");
              System.out.println("Statistic report: ");
              long avgTime;
              long avgTotalTime = 0;
              for (int i = 1; i <= numberOfThreads; i++)
                   avgTotalTime += accessor[i - 1].getAccessTime();
                   avgTime = accessor[i - 1].getAccessTime() / numberOfAccesses;
                   System.out.println(
                        "Thread "
                             + i
                             + "... "
                             + accessor[i
                             - 1].getStatus()
                             + " Total access time: "
                             + accessor[i
                             - 1].getAccessTime()
                             + " ms. Avg individual access time: "
                             + avgTime
                             + " ms.");
              avgTotalTime = avgTotalTime / numberOfThreads;
              // Make it an average over all threads.
              System.out.println(
                   "\nThe average total access time over all threads: "
                        + avgTotalTime
                        + " ms.");
    File#2: AccessorThread.java
    import java.util.Hashtable;
    import java.util.HashMap;
    import java.util.Random;
    * Created on May 13, 2003
    * @author llai
    * Threaded class that accesses a hashtable a certain number of times
    * and measures the access time for a certain number of accesses.
    public class AccessorThread extends Thread
         private Object hash;
         private int numberOfElements;
         private boolean focusedAccess;
         private int numberOfAccesses;
         private int startOffset;
         private int endOffset;
         private long seed;
         private long totalAccessTime;
         private String status;
         * Constructor
         * @param h the Hashtable or HashMap
         * @param keyToAccess the key to access for this thread
         * @param numberOfAccesses the number of access times
         * @param startOffset the start offset
         * @param endOffset the end offset
         public AccessorThread(
              Object h,
              int numberOfElements,
              boolean focusedAccess,
              int numberOfAccesses,
              int startOffset,
              int endOffset,
              long seed)
              this.hash = h;
              this.numberOfElements = numberOfElements;
              this.focusedAccess = focusedAccess;
              this.numberOfAccesses = numberOfAccesses;
              this.startOffset = startOffset;
              this.endOffset = endOffset;
              this.seed = seed;
              status = "Status: alive.";
         * @return the total access time.
         public long getAccessTime()
              return totalAccessTime;
         * @return the status of the thread.
         public String getStatus()
              return status;
         * @return the seed.
         public long getSeed()
              return seed;
         * Run method, accesses the hashtable at keyToAccess for
         * numberOfAccesses times, and measures the response time for
         * a certain number of those accesses.
         public void run()
              * Begin accessing the table, time only after startOffset accesses,
              * and only up to (numberOfAccesses - endOffset) accesses.
              * Access one value if focusedAccess on.
              * Access random values if focusedAccess off.
              Random numGenerator = new Random(seed);
              if (focusedAccess)
                   Object keyToAccess =
                        new Integer(numGenerator.nextInt(numberOfElements));
                   // Access without timing for the first startOffset accesses.
                   int i;
                   // Check if hash is a HashMap or Hashtable.
                   if (hash instanceof HashMap)
                        for (i = 0; i < startOffset; i++)
                             ((HashMap) hash).get(keyToAccess);
                        // Now start timing for the accesses.
                        long startTime = System.currentTimeMillis();
                        while (i < numberOfAccesses - endOffset)
                             ((HashMap) hash).get(keyToAccess);
                             i++;
                        long stopTime = System.currentTimeMillis();
                        totalAccessTime = stopTime - startTime;
                        // Finish accessing the remainder of endOffset times if endOffset is not >= (numberOfAccesses - startOffset).
                        while (i < numberOfAccesses)
                             ((HashMap) hash).get(keyToAccess);
                             i++;
                   else if (hash instanceof Hashtable)
                        for (i = 0; i < startOffset; i++)
                             ((Hashtable) hash).get(keyToAccess);
                        // Now start timing for the accesses.
                        long startTime = System.currentTimeMillis();
                        while (i < numberOfAccesses - endOffset)
                             ((Hashtable) hash).get(keyToAccess);
                             i++;
                        long stopTime = System.currentTimeMillis();
                        totalAccessTime = stopTime - startTime;
                        // Finish accessing the remainder of endOffset times if endOffset is not >= (numberOfAccesses - startOffset).
                        while (i < numberOfAccesses)
                             ((Hashtable) hash).get(keyToAccess);
                             i++;
              // Or else !focusedAccess, therefore access random values.
              else
                   * Get random keys to access and store in an array of Integer objects.
                   * Limit the array to 1000 integer objects.
                   int numberOfRandomInts = numberOfAccesses;
                   if (numberOfAccesses > 1000)
                        numberOfRandomInts = 1000;
                   Integer keysToAccess[] = new Integer[numberOfRandomInts];
                   for (int q = 0; q < numberOfRandomInts; q++)
                        keysToAccess[q] =
                             new Integer(numGenerator.nextInt(numberOfElements));
                   int i;
                   if (hash instanceof HashMap)
                        for (i = 0; i < startOffset; i++)
                             ((HashMap) hash).get(keysToAccess[i % numberOfRandomInts]);
                        // Now start timing for the accesses.
                        long startTime = System.currentTimeMillis();
                        while (i < numberOfAccesses - endOffset)
                             ((HashMap) hash).get(keysToAccess[i % numberOfRandomInts]);
                             i++;
                        // Stop the timing.
                        long stopTime = System.currentTimeMillis();
                        totalAccessTime = stopTime - startTime;
                        // Finish accessing the remainder of endOffset times if endOffset is not >= (numberOfAccesses - startOffset).
                        while (i < numberOfAccesses)
                             ((HashMap) hash).get(keysToAccess[i % numberOfRandomInts]);
                             i++;
                   else if (hash instanceof Hashtable)
                        for (i = 0; i < startOffset; i++)
                             ((Hashtable) hash).get(keysToAccess[i % numberOfRandomInts]);
                        // Now start timing for the accesses.
                        long startTime = System.currentTimeMillis();
                        while (i < numberOfAccesses - endOffset)
                             ((Hashtable) hash).get(keysToAccess[i % numberOfRandomInts]);
                             i++;
                        // Stop the timing.
                        long stopTime = System.currentTimeMillis();
                        totalAccessTime = stopTime - startTime;
                        // Finish accessing the remainder of endOffset times if endOffset is not >= (numberOfAccesses - startOffset).
                        while (i < numberOfAccesses)
                             ((Hashtable) hash).get(keysToAccess[i % numberOfRandomInts]);
                             i++;
              status = "Status: dead.";
    # File #3: run.sh
    echo Compiling the program...
    javac Driver.java AccessorThread.java
    export CLASSPATH=.
    echo Running the program...
    java Driver

  • Java Performance: Linux vs Windows, 64 bit vs 32 bit JVM

    I am looking for information about how Java performs on Linux vs Windows and how the results are affected when a 64 bit jvm is used.
    Disk access or IO are not important, I would really like to see some benchmarks on floating point arithmetic and/or heavy memory access, assuming that on all environments enough physical memory could be addressed so the extra available memory space for 64 bit os'es/jvm's is not important.
    I have been searching on Google but didn't find any useful recent results, only some very old stuff.
    Does anybody know how the OS and the JVM affect performance? I could imagine that Java VM's are a little less optimized on Linux because of the fact that Linux has a smaller marketshare, but maybe that is not true?
    Also I could imagine that 64 bit JVMs could be faster because they could use more cpu registers, on the other hand they could slow down a little because all memory pointers are twice as long so more data should be processed?
    Any comments are greatly appreciated!

    On the Linux vs Windows theme, you would probably find out, after much browsing & research, that there is no difference in performance or that the difference is too small to be even considered.
    On the 32-bit vs 64-bit subject, here's a recent blog that you could look at:
    http://dev2dev.bea.com/blog/patel_jayesh_j/archive/2007/09/32bit_or_64bit.html
    That blog has some links to some benchmarks, but here again, the differences are unlikely to be earth-shattering.
    Not much to add here, other than the most important improvement in performance derive from how the application is designed, and in some cases, how the JVM heap itself is configured, independently on Linux vs Win or 32 vs 64 -bit considerations.
    In a case when you would really need to know, there is no general right answer to your question, and you'd have to benchmark your specific application yourself in your specific execution environment to be able to evaluate possible benefits of 64-bit vs 32-bit.
    In conclusion, I wouldn't waste much time on all this, and you probably have better things to do yourself.

  • EVA performance with ESXi 5.1

    We are having problems with some of our Linux VMs going read only and Windows VMs slowing right down on our P6000 EVA. The problem tends to appear during a backup window, but sometimes during the working day under heavy periods of activity. It seems to be storage performance related. ESXi 5.1 is using round robin and there are no network issues.
    Has anyone tried these advanced ESXi settings which are reported to improve performance?
    (From: http://arritdor.e-wilkin.com/2012/05/vmware-esxi-and-hp-3par-storage.html)
    DiskMaxIOSize   = 128
    QFullSampleSize = 32  
    QFullThreshold  = 4

    I think it might be beneficial to take a look at the array performance statistics. The evaperf can be found at c:\Program Files\Hewlett-Packard\EVA Performance Monitor if it is selected during CommandView installation. Some useful stats to collect are:
    evaperf cs –cont 60 -dur 3600 -csv -ts2 -sz xxxx-xxxx-xxxx-xxxx > controller.csv
    evaperf hps –cont 60 -dur 3600 -csv -ts2 -sz xxxx-xxxx-xxxx-xxxx > hostport.csv
    evaperf vd –cont 60 -dur 3600 -csv -ts2 -sz xxxx-xxxx-xxxx-xxxx > vdisk.csv
    evaperf pdg –cont 60 -dur 3600 -csv -ts2 -sz xxxx-xxxx-xxxx-xxxx > diskgroup.csv
    evaperf pda –cont 60 -dur 3600 -csv -ts2 -sz xxxx-xxxx-xxxx-xxxx > disk.csv
    where xxxx-xxxx-xxxx-xxxx is the world wide name of the array e.g. 5000–1FE1–5000–A9F0. You can probably find it on the controller LCD or some serial number label sticker at the back.
    Because window server command prompt does not support running command at the background (at least I do not know how) unlike hp-ux server, you will have to open several command prompt window and run these command at the same time. Then probably zip and posted it somewhere so that we can all take a look.
    the 60 here means we make one collection every 60 seconds, and -dur 3600 means for a duration of 3600 seconds which is one hour. For a start, we do not want to collect huge amount of data. We probably want to collect one set when array has no problem, and one set when array has problem, then make a comparison.

Maybe you are looking for