Auto-indexing is slow for arrays with 1 dimensions
Hi,
I was looking at the performance of operations on all individual elements in 3D arrays, especially the difference between auto-indexing (left image) and manual-indexing (right image, calling "Index array" and "Replace array subset" in the innermost loop). I'm a bit puzzled by the results and post it here for discussion and hopefully somebody's benefit in the future.
Left: auto-indexing; right: manual-indexing
In the tests I added a different random number to all individual elements in a 3D array. I found that auto-indexing is 1.2 to 1.5 times slower than manual-indexing. I also found that the performance of auto-indexing is much more dependent on the size the dimensions: an array with 1000x200x40 elements is 20% slower than an array with 1000x40x200 elements. For manual-indexing there is hardly any difference. The detailed results can be found at the end of this post.
I was under the impression that auto-indexing was the preferred method for this kind of operation: it achieves exactly the same result and it is much clearer in the block diagram. I also expected that auto-indexing would have been optimized in LabView, but the the tests show this is clearly not the case.
What is auto-indexing doing?
With two tests I looked at how auto-index works.
First, I looked if auto-indexing reorganizes the data in an inefficient way. To do this I made a method "fake-auto-indexing" which calls "Array subset" and "Reshape array" (or "Index array" for a 1D-array) when it enters _every_ loop and calls "Replace array subset" when exiting _every_ loop (as opposed to manual-indexing, where I do this only in the inner loop). I replicated this in a test (calling it fake-auto-indexing) and found that the performance is very similar to auto-indexing, especially looking at the trend for the different array lengths.
Fake-auto-indexing
Second, I looked if Locality of reference (how the 3D array is stored in memory and how efficiently you can iterate over that) may be an issue. Auto-indexing loops over pages-rows-columns (i.e. pages in the outer for-loop, rows in the middle for-loop, columns in the inner for-loop). This can't be changed for auto-indexing, but I did change it for manual and fake-indexing. The performance of manual-indexing is now similar to auto-indexing, except for two cases that I can't explain. Fake-auto-indexing performs way worse in all cases.
It seems that the performance problem with auto-indexing is due to inefficient data organization.
Other tests
I did the same tests for 1D and 2D arrays. For 1D arrays the three methods perform identical. For 2D arrays auto-indexing is 15% slower than manual-indexing, while fake-auto-indexing is 8% slower than manual-indexing. I find it interesting that auto-indexing is the slowest of the three methods.
Finally, I tested the performance of operating on entire columns (instead of every single element) of a 3D array. In all cases it is a lot faster than iterating over individual elements. Auto-indexing is more than 1.8 to 3.4 times slower than manual-indexing, while fake-auto-indexing is about 1.5 to 2.7 times slower. Because of the number of iterations that has to be done, the effect of the size of the column is important: an array with 1000x200x40 elements is in all cases much slower than an array with 1000x40x200 elements.
Discussion & conclusions
In all the cases I tested, auto-indexing is significantly slower than manual-indexing. Because auto-indexing is the standard way of indexing arrays in LabView I expected the method to be highly optimized. Judging by the tests I did, that is not the case. I find this puzzling.
Does anybody know any best practices when it comes to working with >1D arrays? It seems there is a lack of documentation about the performance, surprising given the significant differences I found.
It is of course possible that I made mistakes. I tried to keep the code as straightforward as possible to minimize that risk. Still, I hope somebody can double-check the work I did.
Results
I ran the tests on a computer with a i5-4570 @ 3.20 GHz processor (4 cores, but only 1 is used), 8 GB RAM running Windows 7 64-bit and LabView 2013 32-bit. The tests were averaged 10 times. The results are in milliseconds.
3D-arrays, iterate pages-rows-columns
pages x rows x cols : auto manual fake
40 x 200 x 1000 : 268.9 202.0 268.8
40 x 1000 x 200 : 276.9 204.1 263.8
200 x 40 x 1000 : 264.6 202.8 260.6
200 x 1000 x 40 : 306.9 205.0 300.0
1000 x 40 x 200 : 253.7 203.1 259.3
1000 x 200 x 40 : 307.2 206.2 288.5
100 x 100 x 100 : 36.2 25.7 33.9
3D-arrays, iterate columns-rows-pages
pages x rows x cols : manual fake
40 x 200 x 1000 : 277.6 457
40 x 1000 x 200 : 291.6 461.5
200 x 40 x 1000 : 277.4 431.9
200 x 1000 x 40 : 452.5 572.1
1000 x 40 x 200 : 298.1 460.4
1000 x 200 x 40 : 460.5 583.8
100 x 100 x 100 : 31.7 51.9
2D-arrays, iterate rows-columns
rows x cols : auto manual fake
200 x 20000 : 123.5 106.1 113.2
20000 x 200 : 119.5 106.1 115.0
10000 x 10000 : 3080.25 2659.65 2835.1
1D-arrays, iterate over columns
cols : auto manual fake
500000 : 11.5 11.8 11.6
3D-arrays, iterate pages-rows, operate on columns
pages x rows x cols : auto manual fake
40 x 200 x 1000 : 83.9 23.3 62.9
40 x 1000 x 200 : 89.6 31.9 69.0
200 x 40 x 1000 : 74.3 27.6 62.2
200 x 1000 x 40 : 135.6 76.2 107.1
1000 x 40 x 200 : 75.3 31.2 68.6
1000 x 200 x 40 : 133.6 71.7 108.7
100 x 100 x 100 : 13.0 5.4 9.9
VI's
I attached the VI's I used for testing. "ND_add_random_number.vi" (where N is 1, 2 or 3) is where all the action happens, taking a selector with a method and an array with the N dimensions as input. It returns the result and time in milliseconds. Using "ND_add_random_number_automated_test.vi" I run this for a few different situations (auto/manual/fake-indexing, interchanging the dimensions). The VI's starting with "3D_locality_" are used for the locality tests. The VI's starting with "3D_norows_" are used for the iterations over pages and columns only.
Attachments:
3D_arrays_2.zip 222 KB
Robert,
the copy-thing is not specific for auto-indexing. It is common for all tunnels. A tunnel is first of all a unique data space.
This sounds hard, but there is an optimization in the compiler trying to reduce the number of copies the VI will create. This optimization is called "in-placeness".
The in-placeness algorithm checks, if the wire passing data to the is connected to anything else ("branch"). If there is no other connection then the tunnel, chance is high that the tunnel won't create an additional copy.
Speaking of loops, tunnels always copies. The in-placeness algorithm cannot opt that out.
You can do a small test to elaborate on this: Simply wire "0" (or anything less than the array sizy of the input array) to the 'N' terminal.......
Norbert
PS: Auto-indexing is perfect for a "by element" operation of analysis without modification of the element in the array. If you want to modify values, use shift registers and Replace Array Subset.
CEO: What exactly is stopping us from doing this?
Expert: Geometry
Marketing Manager: Just ignore it.
Similar Messages
-
Is Java still limited to no long integers for index values?
If you know, what is the reason for that limit? Thanks in advance.Only asked if index values were still limited to int.
In other words, is long now allowed or not?No, long is not allowed. Only unsigned int is allowed. This means your arrays are limited to 2 billion elements. So if you have the 64-bit version of Java and more than 64 GB of memory, this might be of concern. If not int, should be fine.
I realize that indexes have to be positive. That was
not the question. There doesn't seem to be any good
reason for limiting index values to a little over 2
trillion however absurdly large that might seem to2 billion. 2^31, because this is the size of an int, The limit could have been mad eto be long but this would have wasted memory for every array, to allow a limit which is still not possible except with the largest computers. Even, then there is no current practice use for such a large array.
most. And I realize that most of the time it will not be an issue.Not even of use.
To answer my own question, I wrote a simple program
defining an index value as long in a "for"
statement's initialization and when I tried to
compile it I got the error message, "found : long /
required: int" (using JCreator with most recent
version of JDK. Perhaps long was not the best choise. Try int for a loop counter.
In addition to int, I found that "byte" and "short"
work. I was amused to find that "char" also works.These can all be converted to int without loosing precision.
Of course, "boolean", "float" and "double" do not work. And "long" does not work.These cannot be converted to int as simply.
So, clearly the limit against using long still exists, but why?You have to pick a limit. Why make long a limit when BigInteger could be used for arrays larger than 2^63 elements.
So my question is simply: Why does this limit still exist? Is there some "problem" that is being avoided? If so, what?Because there is no practical use for an array larger than 2 biilion elements.
Unless you have the 64-bit version of the JDK, you could would hit an address limit in any case. -
Auto index entry generation for a range of sentences?
Is there any way to mark a range of sentences and associate it with a particular index entry? For a lengthy book, I don't think one can really expect the text to never change, hence regenerating the index needs to continue to work even if a bit of text was added which pushes one of the range of sentences onto the next page.
The closest thing I found was to mark the beginning of the range of sentences with an index entry for "the next 1 paragraph," but this doesn't seem to work. In my example document, http://goldfinches.org/InDesign/Failed%20Index%20Entry.indd the "Dog" entry spans two pages, but only 1 paragraph, and yet the index that is generated only has the first page in it.Greg,
There is no way to say, index from here to here. But you can use any of the range designators. From what you're describing, the most likely to be useful is the one that indexes up to the next instance of a particular paragraph style.
Dave -
Need Help for Array with Object
hey there guys... am trying to complete my project which is to create a Library System. Was able to create a list to show the books available when they select the book and click a borrow button it can print out the book.
what problem i have now is that when a student click borrow... the value how can i transfer to an array inside the object student.
i am usin a main screen (ms) who is controlling all the functions. been trying and trying on this for very long hopefully there will be those who are able to help me out.
my customer screen would be like this... but how can i add in the array for books borrowed
import javax.swing.*;
class Customer
private String name;
private int accNo;
private String password;
private double balance;
private Books borrow[]=new Books[5];
int borrowCount=0;
static int customerCount=0;
private MainScreen ms;
Customer(String n, int no, String p, double b)
name=n;
accNo=no;
password=p;
balance=b;
customerCount++;
JOptionPane.showMessageDialog(null,name +" record created");
display();
/* Trying to Create the Array to store information
public void setStudentBorrow(String a)
borrow[borrowCount]=a;
borrowCount++;
JOptionPane.showMessageDialog(null,"Book Borrowed");
public String getStudentBorrow()
for(int i=0;i<borrowCount-1;i++)
{ return borrow; }
public String getPassword()
{ return password; }
public String getName()
{ return name; }
public int getAccNo()
{ return accNo; }
public double getBalance()
{ return balance; }
public void setName(String n)
{ name=n; }
public void setPassword(String p)
{ password=p; }
public void setBalance(double b)
{ balance=b; }
public void setAccNo(int no)
{ accNo=no; }
public void display()
JOptionPane.showMessageDialog(null,
"\nCutomer Number : "+ customerCount+
"\nName :"+name+
"\nAccount Number: "+accNo+
"\nBalance (RM): "+balance,"Customer record",
JOptionPane.INFORMATION_MESSAGE );Cross Post:
http://forum.java.sun.com/thread.jspa?threadID=779224&messageID=4433689#4433689 -
Auto Email to vendor for followup balance quantity deliver
Hi All,
Auto Email to vendor for followup balance quantity deliver,
I have an requirement that Auto email to vendor for reminder with some fixed days (example : 10 days),
Please suggest me any configurations for this and suggest me how to proceed on this..
ThanksHi,
You can use Urging Reminders to supplier with output type MAHN.
Here you set all the reminders to vendors based on days set up earlier then delivery date or 10 before delivery date 5 days before delivery date.
Refer the below docs it may help you..
Output message type MAHN Dunning - ERP SCM - SCN Wiki
Reminders and Urging Letters (Expediters) - Purchasing (MM-PUR) - SAP Library -
Convert "1D array of dynamic data" to "Dynamic data" after auto-indexing with loop
Hi -
I have a question about how exaclty auto-indexing works with the Dynamic Data type when exiting a while loop. I'd like to use the DAQ assistant to record data through an analog input (on the MyDAQ) and then replay that data through an analog output. This works fine if I use the DAQ Assistant to record data for a set amount of time:
However, I run into trouble when I try to use a While loop with a stop button to record data for an arbitrary amount of time. I set the dynamic data tunnel out of the While loop to "Indexing", but then when it exits the loop, it becomes "1-D Array of Dynamic Data" instead of just "Dynamic Data", and get a broken wire when I try to connect it to the input of the DAQ Assistant. I've also tried converting the dynamic data to Array and Waveform data types inside the loop, but have the same issue (they become 2D Array and 2D Array of Waveform respectively when leaving the loop). Try as I might using blocks like Convert to/from Dynamic Data Type, or Array Subset, I'm unable to get them back down to just Dynamic Data or Waveform, which will be accepted by the DAQ Assistant:
One other small note - I'm not sure if using the "Build Array" function with a shift register accomplishes exactly the same thing as just auto-indexing an array out of the loop - so I've tried both, but still have the broken wire issue.
I'm assuming this is just a simple issue of using the right block to convert "1D Array of [Something]" to just plain [Something]. Any help appreciated.
Attachments:
PWM_FlatSequence.vi 101 KB
PWM_FlatSequence_While.vi 129 KBI have been facing problem for all temperature values placing the Write to Measurement File outside the Loop.
In Error message it says" Source: 1D array of dynamic data and Sink: Dynamic data".
Is there any means of convering 1D array of dynamic data to dynamic data?
I would highly appreciate any help.
Attachments:
Temperature Logger.vi 110 KB -
Creating an array without using "auto-indexing"
Hello,
it is terrible! I'm new in working with LV5.0 (Evaluation) and I can't
find any examples how to write data into an array without using the
function "auto-indexing" of the "WHILE" or "FOR"-loops..
The values do not appear all at the same time, so I can't use the
"create_array" function.
So I want to "collect" all occured values and then write them e.g. in a
file.
Did I miss anything in the description ???
Who can help me?
Thanks, CHZCHZ wrote in message news:[email protected]..
> Hello,
> it is terrible! I'm new in working with LV5.0 (Evaluation) and I can't
> find any examples how to write data into an array without using the
> function "auto-indexing" of the "WHILE" or "FOR"-loops..
> The values do not appear all at the same time, so I can't use the
> "create_array" function.
> So I want to "collect" all occured values and then write them e.g. in a
> file.
The "Create array" function can be configured to take arrays as well as
elements. You have to use a loop that iterates as often as necessary,
checking for new data. When it receives new data, it adds the new data to
the existing array. It's probably easier to attach a small example than to
describe it in more
detail.
[Attachment Array.zip, see below]
Attachments:
Array.zip 10 KB -
Using a For Loop Index for Array Element Number?
Hi. Thanks in advance for the help. I can't seem to find what I want to do here using the search function... probably because I don't know how to frame the question appropriately. Ignore the title to this post cause it's probably wrong vocabulary for what I want to do.
Here's the situation:
I have a 2D array where the first column is the X data and each subsequent column is Y data (call it Y1...Yn). The n-value varies from run to run (some runs have 4 columns of data, some have 20), but that is easily gathered when reading the data file. I need to perform a number of operations on each set of data, the least of which is smoothing the data and graphing it, so I'm limiting my discussion to these 2 operations.
What I want is a for loop structure where the indexing of the loop tracks the Y1...Yn columns, grabs each column for the iteration, performs the relevant analysis, and spits out the result, but in a stackable manner. Thus the title; I want to use the for loop's index to mark the array element for building a new array.
See the image attached. A 2D array of 9 columns (X, Y1...Y8) is analyzed such that each data set, (X, Y1), (X, Y2)...(X, Y8) is bundled, graphed, run through a B-spline Fit, of which is also graphed. I need to replace this with something that looks like the for loop structure shown.
I just don't know how to get those two data bundles [(X, Yi) and it's smoothed pair] out of the for loop in a stacked set from every iteration of the for loop.
Again the title, I think I want to build a new array where i use the index of the for loop to control the entries of the new array. But I can't see how to do that.
Any help would be appreciated.
Attachments:
NeedForLoopForThis.PNG 30 KBHello H.R. Dunham, and welcome to the forum!
It seems that you may be looking for Auto-Indexing, a basic feature of loops in LabVIEW. You'll need to transpose your array before wiring it into the for loop, as elements are auto-indexed by row before column, but at that point you should be able to operate on each column and output an array of n cluster elements corresponding to your input columns. Auto-indexing tunnels look like brackets to indicate each element will be indexed automatically- this should be the default when wiring an array into a for loop or when wiring anything out of a for loop.
As for how to build your pairs, I suggest removing the X column and creating a "starter" cluster containing your X data and placeholder Y data before entering your "Y" processing loop. Use the bundle by name function in the for loop to insert your processed column data into the cluster and auto-index the cluster output. Flow would be something like this:
1) Gather data
2) Split X and Y using standard array operations
3) Create "template" cluster with shared X data
4) Pass template cluster and Y-column array into an auto-indexed for loop.
5) Insert processed Y data into cluster inside loop.
6) Auto-index cluster data out of loop.
This tutorial is probably also a good place to get started:
Getting Started with NI LabVIEW Module 3: Loops
http://www.ni.com/white-paper/7528/en/
Hope that helps!
Regards,
Tom L. -
Bug in conditional for loop with an empty array
There appears to be a bug in the for loop with a conditional terminal.
If an empty array is wired to an auto-indexed array input tunnel, an output array tunnel has one element instead of zero.
The array constant on the left is empty.
Top loop without the conditional terminal produces an empty array.
The bottom loop with a never true conditional terminal produces an array with one element
Using LabVIEW version 8.5
Message Edited by TrevMrgn on 05-05-2009 02:01 PM
Solved!
Go to Solution.
Attachments:
For loop bug.png 3 KBCreating the example from scratch, I do not see the error.
Neither do I.
If I add one or more elements to the array (constant or control), then delete them all (using 'Delete Element') the bottom array has one element.
Nup. Still empty.
If I empty the array (using 'Empty Array') both are empty.
Yep.
Also if I create an empty array using 'initialise array' with zero elements, I get one element.
Nup. Still empty.
'Show constant folding' does not appear to change the behaviour, but including a random operation in the loop does.
Agreed that constant folding doesn't cause any odd behaviour. Introducing a random operation (adding two constants) in the loop doesn't either.
I've tried all your methods here Trevor, but I can't replicate this behaviour in 8.5.1 under WinXP SP3. Sorry!
Maybe somebody else will be able to help, or find a useable workaround.
Thoric (CLA, CLED, CTD and LabVIEW Champion) -
I have an early 2011 MacBook Pro which has been running slow for a while. After looking at responses to similar problems I have downloaded and run EtreCheck. Please can someone help me with what it all means.
Thanks in advance.
EtreCheck version: 1.9.15 (52)
Report generated 19 September 2014 08:07:14 GMT+8
Hardware Information: ?
MacBook Pro (13-inch, Early 2011) (Verified)
MacBook Pro - model: MacBookPro8,1
1 2.3 GHz Intel Core i5 CPU: 2 cores
4 GB RAM
Video Information: ?
Intel HD Graphics 3000 - VRAM: 384 MB
Color LCD 1280 x 800
System Software: ?
OS X 10.9.4 (13E28) - Uptime: 0 days 0:4:29
Disk Information: ?
Hitachi HTS545032B9A302 disk0 : (320.07 GB)
S.M.A.R.T. Status: Verified
EFI (disk0s1) <not mounted>: 209.7 MB
Macintosh HD (disk0s2) / [Startup]: 319.21 GB (147 GB free)
Recovery HD (disk0s3) <not mounted>: 650 MB
MATSHITADVD-R UJ-898
USB Information: ?
Apple Inc. FaceTime HD Camera (Built-in)
Apple Inc. BRCM2070 Hub
Apple Inc. Bluetooth USB Host Controller
Apple Inc. Apple Internal Keyboard / Trackpad
Apple Computer, Inc. IR Receiver
Thunderbolt Information: ?
Apple Inc. thunderbolt_bus
Gatekeeper: ?
Mac App Store and identified developers
Kernel Extensions: ?
[not loaded] com.seagate.driver.PowSecDriverCore (5.2.4 - SDK 10.4) Support
[not loaded] com.seagate.driver.PowSecLeafDriver_10_4 (5.2.4 - SDK 10.4) Support
[not loaded] com.seagate.driver.PowSecLeafDriver_10_5 (5.2.4 - SDK 10.5) Support
[not loaded] com.seagate.driver.SeagateDriveIcons (5.2.4 - SDK 10.4) Support
[loaded] com.sophos.kext.sav (9.1.55 - SDK 10.7) Support
[loaded] com.sophos.nke.swi (9.1.50 - SDK 10.8) Support
Launch Daemons: ?
[loaded] com.adobe.fpsaud.plist Support
[loaded] com.microsoft.office.licensing.helper.plist Support
[running] com.sophos.autoupdate.plist Support
[running] com.sophos.configuration.plist Support
[running] com.sophos.intercheck.plist Support
[running] com.sophos.notification.plist Support
[running] com.sophos.scan.plist Support
[running] com.sophos.sxld.plist Support
[running] com.sophos.webd.plist Support
[running] com.trusteer.rooks.rooksd.plist Support
Launch Agents: ?
[loaded] com.divx.dms.agent.plist Support
[loaded] com.divx.update.agent.plist Support
[running] com.sophos.uiserver.plist Support
[running] com.trusteer.rapport.rapportd.plist Support
User Launch Agents: ?
[loaded] com.adobe.ARM.[...].plist Support
[running] com.amazon.music.plist Support
[loaded] com.google.keystone.agent.plist Support
[not loaded] jp.co.canon.Inkjet_Extended_Survey_Agent.plist Support
User Login Items: ?
iTunesHelper
TomTomHOMERunner
AdobeResourceSynchronizer
Dropbox
Internet Plug-ins: ?
FlashPlayer-10.6: Version: 15.0.0.152 - SDK 10.6 Support
DivX Web Player: Version: 3.2.1.977 - SDK 10.6 Support
AdobePDFViewerNPAPI: Version: 11.0.09 - SDK 10.6 Support
AdobePDFViewer: Version: 11.0.09 - SDK 10.6 Support
Flash Player: Version: 15.0.0.152 - SDK 10.6 Support
EPPEX Plugin: Version: 10.0 Support
Default Browser: Version: 537 - SDK 10.9
OVSHelper: Version: 1.1 Support
QuickTime Plugin: Version: 7.7.3
SharePointBrowserPlugin: Version: 14.4.4 - SDK 10.6 Support
iPhotoPhotocast: Version: 7.0 - SDK 10.7
Safari Extensions: ?
Ultimate
Audio Plug-ins: ?
BluetoothAudioPlugIn: Version: 1.0 - SDK 10.9
AirPlay: Version: 2.0 - SDK 10.9
AppleAVBAudio: Version: 203.2 - SDK 10.9
iSightAudio: Version: 7.7.3 - SDK 10.9
iTunes Plug-ins: ?
Quartz Composer Visualizer: Version: 1.4 - SDK 10.9
3rd Party Preference Panes: ?
Flash Player Support
Perian Support
Trusteer Endpoint Protection Support
Time Machine: ?
Skip System Files: NO
Auto backup: YES
Volumes being backed up:
Macintosh HD: Disk size: 297.29 GB Disk used: 160.38 GB
Destinations:
Data [Network] (Last used)
Total size: 2 TB
Total number of backups: 99
Oldest backup: 2012-04-20 17:05:32 +0000
Last backup: 2014-09-18 23:49:25 +0000
Size of backup disk: Excellent
Backup size 2 TB > (Disk size 297.29 GB X 3)
Time Machine details may not be accurate.
All volumes being backed up may not be listed.
Top Processes by CPU: ?
6% InterCheck
5% iCalExternalSync
3% WindowServer
2% CalendarAgent
2% SystemUIServer
Top Processes by Memory: ?
152 MB SophosScanD
147 MB InterCheck
106 MB SophosAntiVirus
66 MB Dropbox
57 MB com.apple.iTunesLibraryService
Virtual Memory Information: ?
161 MB Free RAM
1.55 GB Active RAM
1.41 GB Inactive RAM
902 MB Wired RAM
611 MB Page-ins
0 B Page-outsUninstall Trusteer software
http://www.trusteer.com/support/uninstalling-rapport-mac-os-x
Remove Sophos
https://discussions.apple.com/message/21069437#21069437 -
Problem description:
My MacBook Pro is running really slow. Applications won’t open or take up to 10 min.
EtreCheck version: 2.1.5 (108)
Report generated December 27, 2014 at 9:15:58 PM CST
Click the [Support] links for help with non-Apple products.
Click the [Details] links for more information about that line.
Click the [Adware] links for help removing adware.
Hardware Information: ℹ️
MacBook Pro (13-inch, Late 2011) (Verified)
MacBook Pro - model: MacBookPro8,1
1 2.4 GHz Intel Core i5 CPU: 2-core
4 GB RAM Upgradeable
BANK 0/DIMM0
2 GB DDR3 1333 MHz ok
BANK 1/DIMM0
2 GB DDR3 1333 MHz ok
Bluetooth: Old - Handoff/Airdrop2 not supported
Wireless: en1: 802.11 a/b/g/n
Video Information: ℹ️
Intel HD Graphics 3000 - VRAM: 384 MB
Color LCD 1280 x 800
System Software: ℹ️
OS X 10.10.1 (14B25) - Uptime: one day 6:55:8
Disk Information: ℹ️
TOSHIBA MK5065GSXF disk0 : (500.11 GB)
EFI (disk0s1) <not mounted> : 210 MB
Recovery HD (disk0s3) <not mounted> [Recovery]: 650 MB
HD (disk1) / [Startup]: 498.88 GB (5.04 GB free) (Low!)
Core Storage: disk0s2 499.25 GB Online
MATSHITADVD-R UJ-8A8
USB Information: ℹ️
Apple Inc. FaceTime HD Camera (Built-in)
Apple Inc. BRCM2070 Hub
Apple Inc. Bluetooth USB Host Controller
Apple Inc. Apple Internal Keyboard / Trackpad
Apple Computer, Inc. IR Receiver
Thunderbolt Information: ℹ️
Apple Inc. thunderbolt_bus
Gatekeeper: ℹ️
Mac App Store and identified developers
Kernel Extensions: ℹ️
/System/Library/Extensions
[loaded] com.epson.driver.EPSONProjectorUDAudio (1.30 - SDK 10.4) [Support]
[not loaded] com.seagate.driver.PowSecDriverCore (5.2.6 - SDK 10.4) [Support]
/System/Library/Extensions/Seagate Storage Driver.kext/Contents/PlugIns
[not loaded] com.seagate.driver.PowSecLeafDriver_10_4 (5.2.6 - SDK 10.4) [Support]
[not loaded] com.seagate.driver.PowSecLeafDriver_10_5 (5.2.6 - SDK 10.5) [Support]
[not loaded] com.seagate.driver.SeagateDriveIcons (5.2.6 - SDK 10.4) [Support]
Problem System Launch Agents: ℹ️
[killed] com.apple.CallHistoryPluginHelper.plist
[killed] com.apple.coreservices.appleid.authentication.plist
[killed] com.apple.icloud.fmfd.plist
[killed] com.apple.telephonyutilities.callservicesd.plist
4 processes killed due to memory pressure
Problem System Launch Daemons: ℹ️
[killed] com.apple.ctkd.plist
[killed] com.apple.icloud.findmydeviced.plist
[killed] com.apple.ifdreader.plist
[killed] com.apple.nehelper.plist
[killed] com.apple.wdhelper.plist
[running] com.seagate.TBDecorator.plist [Support]
5 processes killed due to memory pressure
Launch Agents: ℹ️
[not loaded] com.adobe.AAM.Updater-1.0.plist [Support]
[loaded] com.carbonite.launchd.carbonitealerts.plist [Support]
[running] com.carbonite.launchd.carbonitestatus.plist [Support]
[loaded] com.coupons.coupond.plist [Support]
[loaded] com.hp.help.tocgenerator.plist [Support]
[loaded] com.trendmicro.itis.dca.plist [Support]
[running] com.trendmicro.itis.uimgmt.agent.plist [Support]
Launch Daemons: ℹ️
[failed] com.adobe.fpsaud.plist [Support]
[running] com.carbonite.launchd.carbonitedaemon.plist [Support]
[loaded] com.microsoft.office.licensing.helper.plist [Support]
[running] com.trendmicro.icore.av.plist [Support]
[running] com.trendmicro.icore.main.plist [Support]
[running] com.trendmicro.icore.wp.plist [Support]
[running] com.trendmicro.itis.plugin.plist [Support]
User Launch Agents: ℹ️
[loaded] com.adobe.AAM.Updater-1.0.plist [Support]
[loaded] com.google.keystone.agent.plist [Support]
[invalid?] com.jdibackup.JustCloud.autostart.plist [Support]
[invalid?] com.jdibackup.JustCloud.backupstart.plist [Support]
[loaded] com.trendmicro.itis.uninstaller.plist [Support]
User Login Items: ℹ️
iTunesHelper Application (/Applications/iTunes.app/Contents/MacOS/iTunesHelper.app)
bomgar-scc-20130819-104735 UNKNOWN (missing value)
bomgar-scc-20130905-143949 UNKNOWN (missing value)
USB Display Agent Application (/Applications/USB Display/USB Display.app/Contents/Resources/USB Display Agent.app)
WDDriveUtilityHelper Application (/Applications/WD Drive Utilities.app/Contents/Resources/WDDriveUtilityHelper.app)
WDSecurityHelper Application (/Applications/WD Security.app/Contents/Resources/WDSecurityHelper.app)
USB Display Agent Application (/Applications/USB Display/USB Display.app/Contents/Resources/USB Display Agent.app)
bomgar-scc-20130819-104735 UNKNOWN (missing value)
bomgar-scc-20130905-143949 UNKNOWN (missing value)
HP Scheduler Application (/Library/Application Support/Hewlett-Packard/Software Update/HP Scheduler.app)
HP Product Research Application (/Library/Application Support/Hewlett-Packard/Customer Participation/HP Product Research.app)
Internet Plug-ins: ℹ️
SharePointBrowserPlugin: Version: 14.4.7 - SDK 10.6 [Support]
FlashPlayer-10.6: Version: 15.0.0.223 - SDK 10.6 [Support]
CouponPrinter-FireFox_v2: Version: 5.0.5 - SDK 10.6 [Support]
Flash Player: Version: 15.0.0.223 - SDK 10.6 Mismatch! Adobe recommends 16.0.0.235
QuickTime Plugin: Version: 7.7.3
Default Browser: Version: 600 - SDK 10.10
User internet Plug-ins: ℹ️
npBcsMcTcIO: Version: Unknown [Support]
Safari Extensions: ℹ️
Trend Micro Toolbar [Installed]
3rd Party Preference Panes: ℹ️
Carbonite [Support]
Flash Player [Support]
Paragon NTFS for Mac ® OS X [Support]
Seagate Dashboard for Mac OSX [Support]
Time Machine: ℹ️
Skip System Files: NO
Mobile backups: ON
Auto backup: YES
Volumes being backed up:
HD: Disk size: 498.88 GB Disk used: 493.84 GB
Destinations:
Jens Seagate Backup [Local]
Total size: 999.86 GB
Total number of backups: 6
Oldest backup: 2014-06-24 10:33:56 +0000
Last backup: 2014-12-17 17:14:18 +0000
Size of backup disk: Too small
Backup size 999.86 GB < (Disk used 493.84 GB X 3)
My Passport for Mac [Local]
Total size: 2.00 TB
Total number of backups: 14
Oldest backup: 2013-08-21 19:19:35 +0000
Last backup: 2014-12-19 11:44:23 +0000
Size of backup disk: Excellent
Backup size 2.00 TB > (Disk size 498.88 GB X 3)
Top Processes by CPU: ℹ️
6% coreaudiod
5% CarboniteDaemon
4% iMovie
3% JustCloud
3% Wondershare Player
Top Processes by Memory: ℹ️
335 MB iPhoto Library Manager
180 MB Finder
142 MB iMovie
70 MB Preview
51 MB Microsoft Word
Virtual Memory Information: ℹ️
149 MB Free RAM
1.04 GB Active RAM
897 MB Inactive RAM
1.17 GB Wired RAM
30.53 GB Page-ins
1.42 GB Page-outs
Diagnostics Information: ℹ️
Dec 26, 2014, 05:17:12 PM /Library/Logs/DiagnosticReports/iPhoto_2014-12-26-171712_[redacted].cpu_resourc e.diag [Details]
Dec 26, 2014, 03:47:17 PM /Library/Logs/DiagnosticReports/CarboniteDaemon_2014-12-26-154717_[redacted].cp u_resource.diag [Details]
Dec 26, 2014, 02:22:01 PM Self test - passed
Dec 26, 2014, 11:48:03 AM /Library/Logs/DiagnosticReports/CarboniteDaemon_2014-12-26-114803_[redacted].cp u_resource.diag [Details]You have nearly run out of disk space. Either purchase a larger one or start cleaning out the one you have?
-
Why is the Star Transformation using two indexes for the same dimension?
Hi,
Recently, I have made an investigation about the Star Transformation feature. I have found a strange test case, which plays an important role in my strategy for our overall DWH architecture. Here it is:
The Strategy:
I would like to have the classical Star Transformation approach (single column Bitmap Indexes for each dimension foreign key column in the fact table), together with additional Bitmap Join Indexes for some of the dimension attributes, which would benefit from the materialization of the join (bitmap merge operation will be skipped/optimized).
The query:
select dp.brand, ds. region_name, dc.region_name
, count(*), sum(f.extended_price)
from fact_line_item f
, dim_part dp
, dim_supplier ds
, dim_customer dc
where dp.mfgr = 10 -- dimension selectivity = 1/10 --> acttual/fact selectivity = 6/10
and f.part_dk = dp.dk
and ds.region_name = 'REGION #1' -- dimension selectivity = 1/9
and f.supplier_dk = ds.dk
and dc.region_name = 'REGION #1' -- dimension selectivity = 1/11
and f.customer_dk = dc.dk
group by dp.brand, ds. region_name, dc.region_name
The actual plan:
| Id | Operation | Name | Starts | E-Rows | Cost (%CPU)| A-Rows | A-Time | Buffers | Reads |
| 0 | SELECT STATEMENT | | 1 | | 3247 (100)| 1 |00:01:42.05 | 264K| 220K|
| 1 | HASH GROUP BY | | 1 | 2 | 3247 (1)| 1 |00:01:42.05 | 264K| 220K|
|* 2 | HASH JOIN | | 1 | 33242 | 3037 (1)| 217K|00:01:29.67 | 264K| 220K|
|* 3 | TABLE ACCESS FULL | DIM_SUPPLIER | 1 | 1112 | 102 (0)| 1112 |00:00:00.01 | 316 | 4 |
|* 4 | HASH JOIN | | 1 | 33245 | 2934 (1)| 217K|00:01:29.10 | 264K| 220K|
|* 5 | TABLE ACCESS FULL | DIM_CUSTOMER | 1 | 910 | 102 (0)| 910 |00:00:00.08 | 316 | 8 |
|* 6 | HASH JOIN | | 1 | 33248 | 2831 (1)| 217K|00:01:28.57 | 264K| 220K|
|* 7 | TABLE ACCESS FULL | DIM_PART | 1 | 10 | 3 (0)| 10 |00:00:00.01 | 6 | 0 |
| 8 | PARTITION RANGE ALL | | 1 | 36211 | 2827 (1)| 217K|00:01:28.01 | 264K| 220K|
| 9 | TABLE ACCESS BY LOCAL INDEX ROWID| FACT_LINE_ITEM | 6 | 36211 | 2827 (1)| 217K|00:01:33.85 | 264K| 220K|
| 10 | BITMAP CONVERSION TO ROWIDS | | 6 | | | 217K|00:00:07.09 | 46980 | 3292 |
| 11 | BITMAP AND | | 6 | | | 69 |00:00:08.33 | 46980 | 3292 |
| 12 | BITMAP MERGE | | 6 | | | 193 |00:00:02.09 | 2408 | 1795 |
| 13 | BITMAP KEY ITERATION | | 6 | | | 4330 |00:00:04.66 | 2408 | 1795 |
| 14 | BUFFER SORT | | 6 | | | 60 |00:00:00.01 | 6 | 0 |
|* 15 | TABLE ACCESS FULL | DIM_PART | 1 | 10 | 3 (0)| 10 |00:00:00.01 | 6 | 0 |
|* 16 | BITMAP INDEX RANGE SCAN | FACT_LI__P_PART_DIM_KEY_BIX | 60 | | | 4330 |00:00:02.11 | 2402 | 1795 |
|* 17 | BITMAP INDEX SINGLE VALUE | FACT_LI__P_PART_MFGR_BJX | 6 | | | 1747 |00:00:06.65 | 890 | 888 |
| 18 | BITMAP MERGE | | 6 | | | 169 |00:00:02.78 | 16695 | 237 |
| 19 | BITMAP KEY ITERATION | | 6 | | | 5460 |00:00:01.56 | 16695 | 237 |
| 20 | BUFFER SORT | | 6 | | | 5460 |00:00:00.02 | 316 | 0 |
|* 21 | TABLE ACCESS FULL | DIM_CUSTOMER | 1 | 910 | 102 (0)| 910 |00:00:00.01 | 316 | 0 |
|* 22 | BITMAP INDEX RANGE SCAN | FACT_LI__P_CUST_DIM_KEY_BIX | 5460 | | | 5460 |00:00:02.07 | 16379 | 237 |
| 23 | BITMAP MERGE | | 6 | | | 170 |00:00:03.65 | 26987 | 372 |
| 24 | BITMAP KEY ITERATION | | 6 | | | 6672 |00:00:02.23 | 26987 | 372 |
| 25 | BUFFER SORT | | 6 | | | 6672 |00:00:00.01 | 316 | 0 |
|* 26 | TABLE ACCESS FULL | DIM_SUPPLIER | 1 | 1112 | 102 (0)| 1112 |00:00:00.01 | 316 | 0 |
|* 27 | BITMAP INDEX RANGE SCAN | FACT_LI__S_SUPP_DIM_KEY_BIX | 6672 | | | 6672 |00:00:02.74 | 26671 | 372 |
The Question:
Why is the Star Transformation using both indexes FACT_LI__P_PART_DIM_KEY_BIX and FACT_LI__P_PART_MFGR_BJX for the same dimension criteria (dp.mfgr = 10)?? The introduction of the additional Bitmap Join Index actually make Oracle to do the work twice !!!
Anybody, any idea ?!?Dom, here it is the plan with the predicates:
| Id | Operation | Name | Starts | E-Rows | Cost (%CPU)| A-Rows | A-Time | Buffers | Reads |
| 0 | SELECT STATEMENT | | 1 | | 3638 (100)| 1 |00:06:41.17 | 445K| 236K|
| 1 | HASH GROUP BY | | 1 | 2 | 3638 (1)| 1 |00:06:41.17 | 445K| 236K|
|* 2 | HASH JOIN | | 1 | 33242 | 3429 (1)| 217K|00:08:18.02 | 445K| 236K|
|* 3 | TABLE ACCESS FULL | DIM_SUPPLIER | 1 | 1112 | 102 (0)| 1112 |00:00:00.03 | 319 | 313 |
|* 4 | HASH JOIN | | 1 | 33245 | 3326 (1)| 217K|00:08:17.47 | 445K| 236K|
|* 5 | TABLE ACCESS FULL | DIM_CUSTOMER | 1 | 910 | 102 (0)| 910 |00:00:00.01 | 319 | 313 |
|* 6 | HASH JOIN | | 1 | 33248 | 3223 (1)| 217K|00:08:16.63 | 445K| 236K|
|* 7 | TABLE ACCESS FULL | DIM_PART | 1 | 10 | 3 (0)| 10 |00:00:00.01 | 6 | 0 |
| 8 | PARTITION RANGE ALL | | 1 | 36211 | 3219 (1)| 217K|00:08:16.30 | 445K| 236K|
| 9 | TABLE ACCESS BY LOCAL INDEX ROWID| FACT_LINE_ITEM | 6 | 36211 | 3219 (1)| 217K|00:08:40.89 | 445K| 236K|
| 10 | BITMAP CONVERSION TO ROWIDS | | 6 | | | 217K|00:00:32.00 | 46919 | 19331 |
| 11 | BITMAP AND | | 6 | | | 69 |00:00:34.50 | 46919 | 19331 |
| 12 | BITMAP MERGE | | 6 | | | 193 |00:00:00.58 | 2353 | 1 |
| 13 | BITMAP KEY ITERATION | | 6 | | | 4330 |00:00:00.10 | 2353 | 1 |
| 14 | BUFFER SORT | | 6 | | | 60 |00:00:00.01 | 6 | 0 |
|* 15 | TABLE ACCESS FULL | DIM_PART | 1 | 10 | 3 (0)| 10 |00:00:00.01 | 6 | 0 |
|* 16 | BITMAP INDEX RANGE SCAN | FACT_LI__P_PART_DIM_KEY_BIX | 60 | | | 4330 |00:00:00.07 | 2347 | 1 |
|* 17 | BITMAP INDEX SINGLE VALUE | FACT_LI__P_PART_MFGR_BJX | 6 | | | 1747 |00:01:23.64 | 882 | 565 |
| 18 | BITMAP MERGE | | 6 | | | 169 |00:00:09.14 | 16697 | 7628 |
| 19 | BITMAP KEY ITERATION | | 6 | | | 5460 |00:00:02.19 | 16697 | 7628 |
| 20 | BUFFER SORT | | 6 | | | 5460 |00:00:00.01 | 316 | 0 |
|* 21 | TABLE ACCESS FULL | DIM_CUSTOMER | 1 | 910 | 102 (0)| 910 |00:00:00.01 | 316 | 0 |
|* 22 | BITMAP INDEX RANGE SCAN | FACT_LI__P_CUST_DIM_KEY_BIX | 5460 | | | 5460 |00:00:08.78 | 16381 | 7628 |
| 23 | BITMAP MERGE | | 6 | | | 170 |00:00:21.46 | 26987 | 11137 |
| 24 | BITMAP KEY ITERATION | | 6 | | | 6672 |00:00:10.29 | 26987 | 11137 |
| 25 | BUFFER SORT | | 6 | | | 6672 |00:00:00.01 | 316 | 0 |
|* 26 | TABLE ACCESS FULL | DIM_SUPPLIER | 1 | 1112 | 102 (0)| 1112 |00:00:00.01 | 316 | 0 |
|* 27 | BITMAP INDEX RANGE SCAN | FACT_LI__S_SUPP_DIM_KEY_BIX | 6672 | | | 6672 |00:00:20.94 | 26671 | 11137 |
Predicate Information (identified by operation id):
2 - access("F"."SUPPLIER_DK"="DS"."DK")
3 - filter("DS"."REGION_NAME"='REGION #1')
4 - access("F"."CUSTOMER_DK"="DC"."DK")
5 - filter("DC"."REGION_NAME"='REGION #1')
6 - access("F"."PART_DK"="DP"."DK")
7 - filter("DP"."MFGR"=10)
15 - filter("DP"."MFGR"=10)
16 - access("F"."PART_DK"="DP"."DK")
17 - access("F"."SYS_NC00017$"=10)
21 - filter("DC"."REGION_NAME"='REGION #1')
22 - access("F"."CUSTOMER_DK"="DC"."DK")
26 - filter("DS"."REGION_NAME"='REGION #1')
27 - access("F"."SUPPLIER_DK"="DS"."DK")
Note
- star transformation used for this statement -
How to create a array with variables dimensions?
I try to create a array like that:
Object[][] data;
data = new Object[] [];
But that's doesn't work!
Apparently I must specify the dimension of my array
So I have done like that :
Object[][] data;
data = new Object[3] [3];
And that work!
But the problem is when I need to add extra elements to my array.
If I write :
data[4][1] = "123";
I have the error message :
java.lang.ArrayIndexOutOfBoundsException
So, how can I defined a array with variables dimensions OR how can I add a dimension to a array?if you have:
Object[][] data;
data = new Object[3] [3];you end yo getting ArrayIndexOutOfBoundException if you try to point to some other Indexes. You can increase the size by doing new:
Object[][] data;
data = new Object[4] [3];and then copy the old Arrays to this one... this is heavy.
Other thing to consider then is using some other datastructure, such as Vector, which grows along you add elements to it.
P_trg -
Frequent index synchronization for table with a lot of data
Hello! I have a table with text column to be indexed. There are a lot of records in the table and the texts in the text column are ofter big enough. The number of records are rising frequently and i need actual search results in any time, so i cant synchronize index 1 or 2 times per day, i need to do it after every insert and i also need to base table not to be locked in anytime i synchronizing index.
So i read the documentation and find a way for me like that:
after every insert on base table do:
ctx_ddl.sync_index('my_index')
And seems like i need to make ctx_ddl.optimize_index() after synchronization.
So will it work fine or i should go another way?
Resuming main goals:
1) I need no locks on base table (or locks on VERY small time);
2) Always actual search results.You can use sync(on commit) in your index parameters, which will make recent DML searchable immediately. You will still need to optimize and rebuild periodically to eliminate index fragmentation. The more frequently that you synchronize, the more fragmented your index will become. Index fragmentation slows down your queries that use the index. If you have some downtime or slow times such as weekends or nights, then that is a good time to do such things. If not, then you may wish to do so "online" to avoid interfering with searches or DML. Depending on the size of your table and the amount of DML, you may choose to optimize and/or rebuild hourly or daily or weekly or monthly or whatever.
-
'Incorrect value type for array element in index N'
Please help. Preflight is flagging multiple 'Incorrect value type for array element in index N" > Required key /F is missing >Array element at index 0 is of wrong type' messages for pages throughout my text document with embedded navigation and annotations.What does it mean? Is there anything I can do?
The origins of the file are from a selection of emails taken from Mail and gmail, and letters probably written on pc (Office) and converted to pdf...
I'm on Mac Mavericks.
Thanks for reading.What problem are you trying to solve with preflighting? What are you checking for?
Maybe you are looking for
-
CS 5.5+: am I just doing theming wrongly?
Hi there Apologies for the long message - I wanted to be clear what I've tried and what my settings are! Any help you can provide is most appreciated. I've built a sample InDesign extension using Extension Builder 2.1 and ActionScript. Although I am
-
Dynamic change of headings in SQL Report
Can you dynamically change a reports Column Heading Text. I want a selected value from the database to become 1 of the column headings.
-
Expert settings for exporting from iMovie
Help. I have a digital video uploaded to iMovie 3.0.3, about 30 minutes in duration and use an external DVD burner (no iDVD). the movie size in iMovie is about 6.3 gigs. I have successfully exported to quicktime but the screen sizes are too small. I
-
Hi Friends, I have a statement as below in one function module : SELECT a~matnr INTO TABLE t_mast FROM mast AS a INNER JOIN stko AS k ON astlnr EQ kstlnr AND astlal EQ kstlal WHERE a~werks eq pla
-
Mail Not Launching When Trying to Open from User Folder
Mail is not launching when I try to open it from within the home folder in my user folder. I have reported the problem to Apple. It does, however, launch when I try to open it from within the Applications folder. Any ideas why? Thanks in advance. Rus