Reading the Index
Hello,
I am running some tests on an index created by Oracle Text and need to know how to extract data from the index.
Specifically, I am trying to determine the longest posting entry (the term that appears in the most documents), the total occurrences of the longest posting entry, and the number of distinct terms in the index.
On a side note I am also unsure on how to determine the size of an index.
Thanks for helping the less experienced!
Hi
Try querying the DR$<index_name>$I table which has the following columns:
TOKEN_TEXT,
TOKEN_TYPE,
TOKEN_FIRST,
TOKEN_LAST,
TOKEN_COUNT,
TOKEN_INFO
TOKEN_COUNT gives you the document frequency (i.e. the number of documents in which a term appears).
The number of rows in the table gives you the number of distinct terms in the index but be careful if you've used theme indexing as these terms are included as well.
By total occurrences, do you mean the total term frequency, i.e. the total number of occurrences of a term across all documents including where a term has multiple occurrences in the same document? If so, I think that information is in the TOKEN_INFO column but it's a BLOB. Has anyone been able to decode it?
Brian
Similar Messages
-
Regular Expressions - How can I read the char of a text using the index?
Assuming that, this is a findWord function which searches in a text and returns the indexes that a given word is located. How to read the characters that are located some pos before each index?
Kind regards,
Vag.
public static void findWord(String type) {
int count = 0;
String regex = "<a word>";
String text = readFile("data/raw/" + type + "/" + type + ".page.all.txt");
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(text);
System.out.print("Word <a word> was found at positions");
while (matcher.find()) {
count++;
System.out.print(", " + matcher.start());
System.out.println(" at a total of " + count + " times!");
}Read the API docs for java.util.regex.Matcher. The matcher object exposes this data via start() and end() methods.
-
The Index Server is in Read Only Mode
Hi,
In our test environment ALL of our jobs are failing. The message in every failed Job Log is:
11/18/05 10:25:15- Starting to run operations (1 total) for job 'Search Update 2 - Run Once'. Will stop on errors. (PID=14064)11/18/05 10:25:16- *** Job Operation #1 of 1: System Agent 'Search Update Agent' [Run as owner 'Administrator']11/18/05 10:25:16- The index server is in read-only mode.11/18/05 10:25:16- The index Server is inaccessible. Search Update cannot continue and will terminate.11/18/05 10:25:16- *** Job Operation #1 failed: Call to perform operation #1 failed with error (0x80044301): (null) (0x4)11/18/05 10:25:16- Call to perform operation #1 failed with error (0x80044301): (null)
has anybody seen this before, and how can I correct it?
Thanks,ShawnThe search server goes into read only mode in response to certain error conditions. Simply restart the service to put the search server back into R/W mode. The server will attempt to self repair on startup. You should review your recent search server logs to see if there are errors that may have caused the transition to read only mode. If problems persist, contact tech support for further assistance in troubleshooting.
-
I am looking for a way to automate index creation using Adobe Reader Pro without having to use the screen user interface, as the indexing has to be run by a batch process.
[discussion moved to Creating, Editing & Exporting PDFs forum.]
-
Reading the stabs.index section of Shared Libs
Hi there.
In order to produce developer-friendly stack-traces, I'd like to read the stabs info from our own projects to map the stack offsets to actual code lines.
So far, I've sucessfully written the reader to handle the lazy stabs inside our object files, and I get the relevant mapping (using N_SLINE, N_FUN and N_SO). But I don't have much luck trying to find these actual .o files from the shared libs (using the N_OBJ stabs contained in the ".stab.index" section of the shared library).
It looks like Sun Studio 11 (maybe earlier versions too) writes the stabs in an odd format in our shared-lib, so that only the beginning of the section is correct (only the one covered by the first N_UNDF) . After a while, the offset to the strings get wrong, even using the "correct" algorithm of adding the offset contained in the previous N_UNDF stab.
So, I can read all the information correctly, except for the actual strings for each stab. I had a look at some other stab-readers (gdb and valgrind), and they suffer from the same issue as my code. Actually, even an objdump -G mySharedlib.so produces the same incorrect output. But obviously, there is a way of find the correct string pointers, as the dumpstabs tool works perfectly.
Does anyone have any pointer to the actual trick needed to read the stabs ? I've already read "The stabs debug format", and the document provided by Sun (and linked from Chris Quenelle's blog): http://developers.sun.com/tools/cc/documentation/ss10_docs/stabs.pdf
CheersFirst of all, I should start by saying that in Sun Studio 12 all the
Sun compilers will generate dwarf as the default debugging format,
and you should be able to use the GNU binutils "addr2line" utility
to query the line number information. The dwarf information produced
by Sun Studio 11 (C compiler, for example) has a glitch that makes
it incompatible with addr2line, but the glitch is fixed in our development
release, and Sun Studio 12 (when it comes out) will work with addr2line.
On Solaris, the addr2line utility is available as "gaddr2line".
But I realize that stabs processing is also useful for the time being.
Here are my ideas for what might be going wrong.
If there are stabs that resulted from COMDAT functions then there are
special formatting issues. The C++ compiler uses COMDAT to implement
templates. The stabs.pdf document describes how COMDAT affects
stabs. Search for the word COMDAT in the document. Because comdat
ELF sections are merged at link-time, it might also help to run
elfdump and dumpstabs on some of the .o files that are being put into
your library as a way of understanding what's going on.
Specifically, the string table offsets need special processing around
any index stabs that result from comdat sections, so I suspect
that's what's causing problems. -
Hi I am building a website using iWeb's white template. I can't write HTML, so can't change the index, but would like to repeat it at the bottom of my pages. Does anyone know which typeface Apple used? Presumably it would be 'web friendly' for use as described above?
Arial
PS. You can see it in the source of the published page where it says new NavBar etc...
".navbar {\n\tfont-family: Arial, sans-serif;\n\tfont-size: 1em;\n\tcolor: #666;\ -
Can I select the data of the indexes?
Hi everybody,
Oracle storages indexes in index type segments as I know.
Is it possible to see the data of these index segments like a table or view?
So I would like to see not only the structue of it with:
select * from dba_segments SG where SG.segment_name = ’index_name’
I think something like this:
For example if I have a table named Table1.
(please don’t see the format of the rowid):
Rowid....field1
1_1.......AA
1_2.......AB
1_3.......BB
and I have an index on field1 named ndx_t1_f1.
Create index ndx_t1_f1 on Table1(field1);
I would like to see something like this
(if it is a simple Btree index like the ndx_t1_f1):
Select * from some_schema.???ndx_t1_f1???
Rowid..........ParentRowid..........ValuePrefix..........SolutionRowid
2_1
2_11............2_1.......................A
2_12............2_1.......................B
2_111..........2_11.....................AA......................1_1
2_112..........2_11.....................AB......................1_2
2_121..........2_12.....................BB......................1_3
And if i run:
select field1 from table1 where filed1 like 'B%'
oracle can use ndx_t1_f1 index like this:
2_1 -> 2_12 -> 2_121 -> BB
Yes, I know, there are some promlems with this imagine, but is it possible, or is it a silly question?
Thanx: lados.Hi,
You asked: „Why do you want to do that?”
Well, I have a table T with fields F1, F2, F3 ...
If I run:
select F2, T.* from T where F1=12345;
the result is one record and the value of F2 is ‘A’.
But if I run:
select F2, F1 from T where F1=12345;
the result is one record and the value of F2 is ‘B’.
And this is the same record. Oppps!!!!!
I think the cause of the problem is the next:
I have an index on F1, F2 named ndx_T_F1_F2
In the second case: oracle uses only the index blocks and no table blocks, because all asked data exists in the index blocks.
In the first case: oracle uses the table blocks too, because I asked all field.
And I think there is a corrupted data on the index.
I created a new index on (F1,F2,F3), and I suggest to use this.
select /*+index(T, ndx_T_F1_F2_F3)*/ T.F2 from T where F1=12345;
the result is one record and the value of F2 is ‘A’.
But what is strange, oracle didn’t realise this inconsistent situation.
(It could be an earlier crash, I don’t know, I’m new at this workplace)
I think, the problem can solve with recreating the index ndx_T_F1_F2.
But it’s a very special situation for me, and I would like to see the corrupted data concretly with my eyes.
I think, that
Select * from some_schema.???ndx_t1_f1???
would I see like a view, but it would based on procedures and not tables.
It would be enough to me, because I only could see the data in the index.
But it is impossible as you wrote, isn’t it?
Thanx: lados. -
The plan doesn't use the index but the cost of INDEX FULL SCAN looks better
Hi,
Well, I'm sure I miss the boat... and if the question is pretty tricky, the answer is probably :"You're stupid Greg!". Well anyway, you'll probably be interested in trying to answer it as I've spent some times on it without any result ! I use Oracle XE on Windows...
1) Below is my query and its plan. You'll find the full schema generation script at the end of this email. Look at the cost (468) of the plan and the cost of the same query when you force the use of the index (116). Why is this ?
select count(distinct col5)
2 from demo
3 where col1 between 1 and 50000
4 and col2=col1
5 and col3=col1
6 and col4=col1;
Plan d'execution
Plan hash value: 2072716547
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 116 | 468 (2)| 00:00:06 |
| 1 | SORT GROUP BY | | 1 | 116 | | |
|* 2 | TABLE ACCESS FULL| DEMO | 1 | 116 | 468 (2)| 00:00:06 |
Predicate Information (identified by operation id):
2 - filter("COL2"="COL1" AND "COL3"="COL1" AND "COL4"="COL1" AND
"COL1"<=50000 AND "COL2"<=50000 AND "COL3"<=50000 AND "COL4"<=50000 AND
"COL1">=1 AND "COL2">=1 AND "COL3">=1 AND "COL4">=1)2) When I force the use of an index (with a Hint), You'll see the cost of the plan is 116 which is definitly better than the TABLE ACCESS FULL (468) :
SQL> l
1 select /*+ index(demo demo_idx)*/ count(distinct col5)
2 from demo
3 where col1 between 1 and 50000
4 and col2=col1
5 and col3=col1
6* and col4=col1
SQL> /
Plan d'execution
Plan hash value: 189561699
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 116 | 437 (2)| 00:00:06 |
| 1 | SORT GROUP BY | | 1 | 116 | | |
| 2 | TABLE ACCESS BY INDEX ROWID| DEMO | 1 | 116 | 437 (2)| 00:00:06 |
|* 3 | INDEX FULL SCAN | DEMO_IDX | 1 | | 436 (2)| 00:00:06 |
Predicate Information (identified by operation id):
3 - filter("COL2"="COL1" AND "COL3"="COL1" AND "COL4"="COL1" AND
"COL1"<=50000 AND "COL2"<=50000 AND "COL3"<=50000 AND "COL4"<=50000 AND
"COL1">=1 AND "COL2">=1 AND "COL3">=1 AND "COL4">=1)3) My question is why is plan1 used while plan2 should be considered better by the optimizer regarding the cost (to make the case even more complex, plan1 is actually more efficient but this is out of the scope of my question. I know that and I know why !).
You'll find a script to generate the structures and data below. I can send you the 10053 traces if you what to go furthermore. Take care the index is a REVERSE index (Don't know if query rewrite should be enabled in order to take advantage of this type of index but it is set to "true" (and "trusted") :
drop table demo;
create table demo (col1 number not null,
col2 number,
col3 number,
col4 number,
col5 varchar2(500));
begin
for i in 1..100000 loop
insert into demo values (i,i,i,i,'This column is used to raise the High Water Mark and '||
' the cost of an TABLE ACCESS FULL operation');
end loop;
end;
commit;
create index demo_idx on demo(col1,col2,col3,col4) reverse;
exec dbms_stats.gather_table_stats(USER, 'DEMO', cascade=>true, -
method_opt=>'FOR ALL COLUMNS SIZE 254', no_invalidate=>false) Any comments are welcome ! Best Regards,
Gregory
Message was edited by:
arkzoyd... I've added the "pre" tagsI suspect this has something to do with db_file_multiblock_read_count
After running provided creation statements by you I got following results:
SQL> show parameter multiblock
NAME TYPE VALUE
db_file_multiblock_read_count integer 16
SQL> set autotrace on
SQL> select count(distinct col5)
2 from demo
3 where col1 between 1 and 50000
4 and col2=col1
5 and col3=col1
6 and col4=col1
7 /
COUNT(DISTINCTCOL5)
1
Execution Plan
Plan hash value: 2072716547
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 116 | 375 (1)| 00:00:05 |
| 1 | SORT GROUP BY | | 1 | 116 | | |
|* 2 | TABLE ACCESS FULL| DEMO | 1 | 116 | 375 (1)| 00:00:05 |
Predicate Information (identified by operation id):
2 - filter("COL2"="COL1" AND "COL3"="COL1" AND "COL4"="COL1" AND
"COL1"<=50000 AND "COL2"<=50000 AND "COL3"<=50000 AND "COL4"<=5000
0 AND
"COL1">=1 AND "COL2">=1 AND "COL3">=1 AND "COL4">=1)
Statistics
196 recursive calls
0 db block gets
1734 consistent gets
850 physical reads
0 redo size
422 bytes sent via SQL*Net to client
385 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
7 sorts (memory)
0 sorts (disk)
1 rows processed
SQL> select /*+ index(demo demo_idx)*/ count(distinct col5)
2 from demo
3 where col1 between 1 and 50000
4 and col2=col1
5 and col3=col1
6 and col4=col1
7 /
COUNT(DISTINCTCOL5)
1
Execution Plan
Plan hash value: 189561699
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| T
ime |
| 0 | SELECT STATEMENT | | 1 | 116 | 431 (1)| 0
0:00:06 |
| 1 | SORT GROUP BY | | 1 | 116 | |
|
| 2 | TABLE ACCESS BY INDEX ROWID| DEMO | 1 | 116 | 431 (1)| 0
0:00:06 |
|* 3 | INDEX FULL SCAN | DEMO_IDX | 1 | | 430 (1)| 0
0:00:06 |
Predicate Information (identified by operation id):
3 - filter("COL2"="COL1" AND "COL3"="COL1" AND "COL4"="COL1" AND
"COL1"<=50000 AND "COL2"<=50000 AND "COL3"<=50000 AND "COL4"<=5000
0 AND
"COL1">=1 AND "COL2">=1 AND "COL3">=1 AND "COL4">=1)
Statistics
1 recursive calls
0 db block gets
50426 consistent gets
428 physical reads
0 redo size
422 bytes sent via SQL*Net to client
385 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1 rows processedNow I modify multiblock_read_count and full scan cost is going up although anyway Oracle by default chooses full scan instead of index access.
SQL> alter session set db_file_multiblock_read_count = 8;
Session altered.
SQL> select count(distinct col5)
2 from demo
3 where col1 between 1 and 50000
4 and col2=col1
5 and col3=col1
6 and col4=col1
7 /
COUNT(DISTINCTCOL5)
1
Execution Plan
Plan hash value: 2072716547
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
| 0 | SELECT STATEMENT | | 1 | 116 | 463 (1)| 00:00:06 |
| 1 | SORT GROUP BY | | 1 | 116 | | |
|* 2 | TABLE ACCESS FULL| DEMO | 1 | 116 | 463 (1)| 00:00:06 |
Predicate Information (identified by operation id):
2 - filter("COL2"="COL1" AND "COL3"="COL1" AND "COL4"="COL1" AND
"COL1"<=50000 AND "COL2"<=50000 AND "COL3"<=50000 AND "COL4"<=5000
0 AND
"COL1">=1 AND "COL2">=1 AND "COL3">=1 AND "COL4">=1)
Statistics
1 recursive calls
0 db block gets
1697 consistent gets
850 physical reads
0 redo size
422 bytes sent via SQL*Net to client
385 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1 rows processed
SQL> select /*+ index(demo demo_idx)*/ count(distinct col5)
2 from demo
3 where col1 between 1 and 50000
4 and col2=col1
5 and col3=col1
6 and col4=col1
7 /
COUNT(DISTINCTCOL5)
1
Execution Plan
Plan hash value: 189561699
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| T
ime |
| 0 | SELECT STATEMENT | | 1 | 116 | 431 (1)| 0
0:00:06 |
| 1 | SORT GROUP BY | | 1 | 116 | |
|
| 2 | TABLE ACCESS BY INDEX ROWID| DEMO | 1 | 116 | 431 (1)| 0
0:00:06 |
|* 3 | INDEX FULL SCAN | DEMO_IDX | 1 | | 430 (1)| 0
0:00:06 |
Predicate Information (identified by operation id):
3 - filter("COL2"="COL1" AND "COL3"="COL1" AND "COL4"="COL1" AND
"COL1"<=50000 AND "COL2"<=50000 AND "COL3"<=50000 AND "COL4"<=5000
0 AND
"COL1">=1 AND "COL2">=1 AND "COL3">=1 AND "COL4">=1)
Statistics
1 recursive calls
0 db block gets
50426 consistent gets
0 physical reads
0 redo size
422 bytes sent via SQL*Net to client
385 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1 rows processedSo I don't know what is the default value of dbfmbrc in XE and not gone too deep to understand how for example system statistics may change your situation.
Gints Plivna
http://www.gplivna.eu
P.S. BTW I used Oracle Database 10g Enterprise Edition Release 10.2.0.1.0.
Message was edited by:
gintsp
listened to Williams suggestion :) -
The device, an Ocean Optics spectrometer in columns of about 9000 cells.I'm saving this as a lvm file using the "write to measurement file.vi". But it doesn't give me the flexibility as far as I can tell.
I need to move the column by the index of the for loop, so that when i = n, the data will take up the n+1 column. (the 1st column is used for wavelength). How do I use the "write to spreadsheet file.vi" to do this? Also, if I use the "write to spreadsheet file.vi", is there a way one can increment the file name, so that the data isn't written over. I like what "write to measurement file.vi" does.
I'd really appreciate any help someone can give me. I'm a novice at this, so the greater the detail, the better. Thanks!!!You cannot write one column at a time to a spreadsheet file, because a file is arranged linearly and adding a column would need to move (=read and rewwrite elsewhere) almost all existing elements to interlace the new data. You can only append new rows without having to touch the already written data.
Fields typically don't have fixed width. An exception would be binary files that are pre-allocated at the final size. In this case you can write columns by setting the file positions for each element. It still will be very inefficient.
What you could do is append rows until all data is written, the read, transpose, and write back the final file.
What you also could to is build the final array in a shift register and write the entire things to file at once after all data is present.
LabVIEW Champion . Do more with less code and in less time . -
I'm having some difficulty with transferring images from a UDP Client to a UDP Server. The issue is receiving the bytes necessary to update an original image sent from the Client to the Server and updating the Servers List(Of Byte) with the
new bytes replacing bytes in that list. This is a simplex connection where the Sever receives and the Client sends to utilize the least amount of bandwidth for a "Remote Desktop" style application where the Server side needs image updates of whatever
occurs on the Client desktop.
So far I can tranfer images with no issue. The images can be be any image type (.Bmp, .Gif, .JPeg, .Png, etc). I was working with sending .JPeg's as they appear to be the smallest size image when a Bitmap is saved to a memory stream as type
.JPeg. And then I am using GZip to compress that byte array again so it is much smaller. However on a loopback on my NIC the speed for sending a full size screen capture is not very fast as the Server updates fairly slowly unless the Clients screen capture
Bitmap is reduced in size to about 1/3'd of the original size. Then about 12000 bytes or less are sent for each update.
Due to .JPeg compression I suppose there is no way to get the difference in bytes between two .JPegs and only send those when something occurs on the desktop that alters the desktop screen capture image. Therefore I went to using .Bmp's as each
.Bmp contains the same number of bytes in its array regardless of the image alterations on the desktop. So I suppose the difference in bytes from a second screen capture and an inital screen capture are what is different in the second image from the initial
image.
What I have done so far is save an initial Bitmap of a screen capture using a memory stream and saving as type .Bmp which takes less than 93 milliseconds for 4196406 bytes. Compressing that takes less than 118 milliseconds to 197325 bytes for
the current windows on the desktop. When that is done PictureBox1 is updated from nothing to the captured image as the PictureBox's background image with image layout zoom and the PictureBox sized at 1/2 my screens width and 1/2 my screens height.
Then I save a new Bitmap the same way which now contains different image information as the PictureBox is now displaying an image so its back color is no longer displayed (solid color Aqua) and the cursor has moved to a different location. The
second Bitmap is also 4196406 in bytes and compressed it was 315473 bytes in size.
I also just found code from this link Converting a Bitmap to a Byte Array (and Byte Array to Bitmap) which gets a byte array
directly from a Bitmap and the size of that is 3148800 for whatever is full screen captured on my laptop. So I should be able to work with smaller byte arrays at some point.
The issue I'm having is that once the Client sends an image of the desktop to the Server I only want to update the server with any differences occuring on the Clients desktop. So what I have done is compare the first screen captures bytes (stored
in a List(Of Byte)) to the second screen captures bytes (stored in a List(Of Byte)) by using a For/Next for 0 to 4196405 where if a byte in the first screen captures List is not equal to a byte in the second screen captures List I add the index and byte of
the second screen captures list to a Dictionary(Of Integer, Byte). The Dictionary then only contains the indexes and bytes that are different between the first screen capture and second screen capture. This takes about 125 milliseconds which I think is pretty
fast for 4196406 byte comparison using a For/Next and adding all the different bytes and indexes for each byte to a Dictionary.
The difference in Bytes between the inital screen capture and the second screen capture is 242587 as an example which changes of course. For that amount of bytes the Dictionary contains 242587 integers as indexes and 242587 bytes as different
bytes totaling 485174 bytes for both arrays (keys, values). Compressed the indexes go from 242587 to 43489 bytes and the values go from 242587 to 34982 bytes. Which means I will have to send 78, 481 bytes from the Client to the Server to update the display
on the server. Quite smaller than the original 4196406 bytes of the second Bitmap saved to type .Bmp or the compressed size of that array which was 315473 bytes. Plus a few bytes I add as overhead so the server knows when an image array ends and how many packets
were sent for the array so it can discard complete arrays if necessary since UDP is lossfull although probably not so much in current networks like it may originally have been when the internet started.
In reality the data from the Client to the Server will mostly be the cursor as it moves and updating the Server image with only a few hundred bytes I would imagine at a time. Or when the cursor selects a Button for example and the Buttons
color changes causing those differences in the original screen capture.
But the problem is if I send the Dictionaries Indexes and Bytes to the Server then I need to update the original Bitmap List(Of Byte) on the server by removing the Bytes in the received informations Index locations array from the Servers Bitmap
List(Of Byte) and replacing those Bytes with the Bytes in the received informations Byte array. This takes so long using a For/Next for however many indexes are in the received informations Index array to update the Bitmap List(Of Byte) on the server using
"Bmp1Bytes.RemoveAt(Index As Integer)" followed by "Bmp1Bytes.Insert(Index As Integer, Item As Byte)" in the For/Next.
I've tried various For/Next statements including using a new List(Of Byte) with If statements so If the the integer for the For/Next ='s the Key in a Dictionary(Of Integer, Byte) using a Counter to provide the Dictionaries Key value then
the Dictionaries byte value will be added to the List(Of Byte) and the counter will increas by one Else the List(Of Byte) adds the original "Bmp1Bytes" byte at that index to the new List(Of Byte). This takes forever also.
I also tried the same For/Next adding to a new Dictionary(Of Integer, Byte) but that takes forever too.
I think I could use RemoveRange and AddRange to speed things up. But I don't know how to retrieve a contiguous range of indexes in the received indexes that need to be updated in the servers "Bmp1Bytes" List(Of Byte) from the received
array of indexes and bytes which are in a Dictionary(Of Integer, Byte). But I believe this would even be slower than some realistic method for replacing all Bytes in a List(Of Byte) when I have the indexes that need to be replaced and the bytes to replace
them with.
Even if I just used AddRange on a new List(Of Byte) to add ranges of bytes from the original "Bmp1Bytes" and the changes from the Dictionary(Of Integer, Byte) I think this would be rather slow. Although I don't know how to do that
by getting contiguous ranges of indexes from the Dictionaries keys.
So I was wondering if there is some method perhaps using Linq or IEnumerable which I've been unable to figure anything out which could do this.
I do have some copy and pasted code which I don't understand how it works that I am using which I would guess could be altered for doing something like this but I can't find information that provides how the code works. Or even if I did
maybe I can't understand it. Like the code below which is extremely fast.
Dim strArray() As String = Array.ConvertAll(Of Integer, String)(BmpComparisonDict.Keys.ToArray, Function(x) x.ToString())
La vida locaMonkeyboy,
That was quite a bit to read, but still a bit unclear. Could you put a specific list of goals/questions, asked in the smallest possible form?
It seems like either you're making a program that monitors activity on your computer, or you're writing some kind of remote pc app.
When you do get your bytes from using lockbits, keep in mind all the files header info would be lost. I think retaining the header info is worth the extra bytes.
The other, thing: I'm not sure if you're taking 32bpp screen shots, but also keep in mind that the "whole desktop" is the final destination for blended graphics, if that makes sense. What I mean is that there is no need to capture an "alpha"
channel for a desktop screenshot, as alpha would always be 255, this could save you 1 byte per pixel captured... Theres nothing "behind" the desktop, therefore no alpha, and every window shown above the desktop is already blended. I suggest using
24Bpp for a full screen capture.
Your X,Y information for the mouse could be stored as UINT16, this would save you a measly 2 bytes per location update/save.
When you update your byte arrays, maybe you can turn the array into a stream and write to whatever index, however many bytes, that should prevent a "Shift" of bytes, and instead overwrite any bytes that "get in the way".
ex
Dim example As String = "This is an example."
Dim insertString As String = "was"
Dim insertBytes As Byte() = System.Text.Encoding.ASCII.GetBytes(insertString)
Dim bytes As Byte() = System.Text.Encoding.ASCII.GetBytes(example)
Dim modifiedBytes As Byte() = {}
Using ms As New System.IO.MemoryStream(bytes)
ms.Position = 5
ms.Write(insertBytes, 0, 3)
modifiedBytes = ms.ToArray
End Using
Dim newString As String = System.Text.Encoding.ASCII.GetString(modifiedBytes)
'Notice how below there isn't the word "is" anymore, and that there isn't a
'space.
'This demonstrates that you overwrite existing data, versus shifting everything to
'the right.
'Returns: This wasan example.
MsgBox(newString)
“If you want something you've never had, you need to do something you've never done.”
Don't forget to mark
helpful posts and answers
! Answer an interesting question? Write a
new article
about it! My Articles
*This post does not reflect the opinion of Microsoft, or its employees.
Well it's too much to read. I was really tired when I wrote it. Even the below is too much to read but perhaps gets the point across of what I would like to do which I think
Joel Engineer may have answered but I'm not sure. As I'm still too tired to understand that yet and research what he said in order to figure it out yet.
But maybe the code below can provide the concept of the operation with the comments in it. But seeing as how I'm still tired it may be confused.
Option Strict On
Imports System.Windows.Forms
Imports System.IO
Imports System.IO.Compression
Imports System.Drawing.Imaging
Imports System.Runtime.InteropServices
Public Class Form1
Dim Bmp1Bytes As New List(Of Byte)
Dim Bmp1BytesCompressed As New List(Of Byte)
Dim Bmp2Bytes As New List(Of Byte)
Dim BmpComparisonDict As New Dictionary(Of Integer, Byte)
Dim BmpDifferenceIndexesCompressed As New List(Of Byte)
Dim BmpDifferenceBytesCompressed As New List(Of Byte)
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
SomeSub()
End Sub
Private Sub SomeSub()
' Pretend this code is in UDP Client app. A screen capture is performed of the desktop. Takes about 90 milliseconds.
Bmp1Bytes.Clear()
Using BMP1 As New Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height)
Using g1 As Graphics = Graphics.FromImage(BMP1)
g1.CopyFromScreen(0, 0, 0, 0, BMP1.Size)
Cursor.Draw(g1, New Rectangle(Cursor.Position.X, Cursor.Position.Y, Cursor.Size.Width, Cursor.Size.Height))
Using MS As New MemoryStream
BMP1.Save(MS, System.Drawing.Imaging.ImageFormat.Bmp)
Bmp1Bytes.AddRange(MS.ToArray)
End Using
End Using
End Using
Bmp1BytesCompressed.AddRange(Compress(Bmp1Bytes.ToArray))
' UDP Client app sends Bmp1BytesCompressed.ToArray to UDP Server which is the entire image of the desktop that the UDP
' Client is on. This image takes awhile to send since compressed it is about 177000 bytes from over 4000000 bytes.
' I will be using different code just to get the bytes from the actual Bitmap in the future. That is not important for now.
' Pretend the UDP Server has received the bytes, decompressed the array received into a List(Of Byte) and is displaying
' the image of the UDP Clients desktop in a PictureBox.
' Now the image on the UDP Clients desktop changes due to the mouse cursor moving as an example. Therefore a new Bitmap
' is created from a screen capture. This takes about 90 milliseconds.
Bmp2Bytes.Clear()
Using BMP2 As New Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height)
Using g1 As Graphics = Graphics.FromImage(BMP2)
g1.CopyFromScreen(0, 0, 0, 0, BMP2.Size)
Cursor.Draw(g1, New Rectangle(Cursor.Position.X, Cursor.Position.Y, Cursor.Size.Width, Cursor.Size.Height))
Using MS As New MemoryStream
BMP2.Save(MS, System.Drawing.Imaging.ImageFormat.Bmp)
Bmp2Bytes.AddRange(MS.ToArray)
End Using
End Using
End Using
' Now I have the original images bytes in Bmp1Bytes and the new images bytes in Bmp2Bytes on the UDP Client. But I don't
' want to send all of the bytes in Bmp2Bytes to the UDP Server. Just the indexes of and the bytes that are different in
' Bmp2Bytes from Bmp1Bytes.
' This takes less than 100 milliseconds for what I've tested so far where over 500000 bytes in Bmp2Bytes are different
' than the bytes in Bmp1Bytes. Usually that amount would be much less. But during testing I was displaying the image
' from Bmp1 bytes in a PictureBox so a large amount of data would change between the first screen shot, the PictureBox
' then displaying an image on the same PC and then the second screen shot.
BmpComparisonDict.Clear()
For i = 0 To Bmp1Bytes.Count - 1
If Bmp1Bytes(i) <> Bmp2Bytes(i) Then
BmpComparisonDict.Add(i, Bmp2Bytes(i))
End If
Next
' So now I have all the difference bytes and their indexes from Bmp2Bytes in the BmpComparisonDict. So I compress
' the indexes into on List and the Bytes into another List.
BmpDifferenceIndexesCompressed.Clear()
BmpDifferenceBytesCompressed.Clear()
BmpDifferenceIndexesCompressed.AddRange(Compress(BmpComparisonDict.Keys.SelectMany(Function(d) BitConverter.GetBytes(d)).ToArray()))
BmpDifferenceBytesCompressed.AddRange(Compress(BmpComparisonDict.Values.ToArray))
' Now pretend the UDP Client has sent both those arrays to the UDP Server which has added both decompressed arrays
' to a Dictionary(Of Integer, Byte). And the server has the original image decompressed bytes received in a List
' called Bmp1Bytes also.
' This is where I am stuck. The UDP Server has the Dictionary. That part was fast. However there is no
' fast method I have found for creating a new List(Of Byte) where bytes in the originally received List(Of Byte) that
' do not have to be altered are placed into a new List(Of Byte) except for the indexes listed in the
' Dictionary(Of Integer, Byte) that need to be placed into the appropriate index locations of the new List(Of Byte).
' The below example for doing so is exceptionally slow. Pretend UpdateDictionary has all of the decompressed indexes
' and bytes received by the UDP Server for the update contained within it.
Dim UpdateDictionary As New Dictionary(Of Integer, Byte)
Dim UpdatedBytes As New List(Of Byte)
Dim Counter As Integer = 0
For i = 0 To Bmp1Bytes.Count - 1
If i = UpdateDictionary.Keys(Counter) Then ' Provides the index contained in the Keys for the Dictionary
UpdatedBytes.Add(UpdateDictionary.Values(Counter))
Counter += 1
If Counter > UpdateDictionary.Count - 1 Then Counter = 0
Else
UpdatedBytes.Add(Bmp1Bytes(i))
End If
Next
' So what I'm trying to do is find an extremely fast method for performing something similar to what the
' above operation performs.
End Sub
Private Function Compress(BytesToCompress() As Byte) As List(Of Byte)
Dim BytesCompressed As New List(Of Byte)
Using compressedStream = New MemoryStream()
Using zipStream = New GZipStream(compressedStream, CompressionMode.Compress)
zipStream.Write(BytesToCompress, 0, BytesToCompress.Count)
zipStream.Close()
BytesCompressed.AddRange(compressedStream.ToArray)
End Using
End Using
Return BytesCompressed
End Function
Private Function Decompress(BytesToDecompress() As Byte) As List(Of Byte)
Dim BytesDecompressed As New List(Of Byte)
Using DecompressedStream = New MemoryStream()
Using zipStream = New GZipStream(DecompressedStream, CompressionMode.Decompress)
zipStream.Write(BytesToDecompress, 0, BytesToDecompress.Count)
zipStream.Close()
BytesDecompressed.AddRange(DecompressedStream.ToArray)
End Using
End Using
Return BytesDecompressed
End Function
End Class
La vida loca -
Extract text file from a folder and read the content
Hi
I have "n" no.of text files saved in a folder with automatically generated naming convention which include DD/MM/YYYY and also some measurement output value.
Eg: 1) Die_1_DUT_outputvalue_DD_MM_YYYY.txt
2) Die_1_DUT_outputvalue_DD_MM_YYYY_ABC.txt
In the above files part of the 2nd file naming convention same as the first file (i.e. Die_1_DUT and DD_MM_YYYY) whereas outputvalue is different and an additional string named ABC is appended.
Now I want to search the 2nd file based on matching the naming pattern with the 1st file (note: the outputvalue in the file name is different for both files) and so far followed this method
1) Use a list folder with *.txt pattern to search all the text files and the output is a 1D array
2) then use array to cluster and then flatten to XML function to have all the text file names as a string element (not 1D array)
3) then pass the output of the 2nd step to the sting match pattern and use a regular expression to get the required file name
4) send the output of the 3rd step to search 1D array to get the index and then get the file name and later use read text file to read the content of the text file
And I am stuck at the 3rd step while sending an input as the regular expression to match the pattern as the outputvalue in the namming convention of the above two files is different is there any way I can actually extract the filename/file?
Any suggestions?
Attachments:
1.png 11 KBSome bits in your code are unnecessary, a leaner implementation here:
Beginner? Try LabVIEW Basics
Sharing bits of code? Try Snippets or LAVA Code Capture Tool
Have you tried Quick Drop?, Visit QD Community. -
Not able to cancel the indexing job in BO Explorer.
Not able to cancel the indexing job in BO Explorer. When I clicked Cancel Indexing, Its still in processing mode. Please advise. Thanks!!
Thanks Henry. Actually Indexing server was down, So it was not getting cancelled. I restarted the Explorer services and it worked fine afterwards.
However, Could you please also let me know if there's any way in Explorer to show description of the Dimension/Measure Objects while exploring data in any information view.
I have defined the object's descriptions in my underlying universe and they are also while confirguring the information space. But I am not able to see the while exploring data.
Could you please advise on this. Thanks!! -
The index tabs were on some kind of toolbar just below the bookmark toolbar and you could see what was open in multiple tabs. I'm not referring to rolling over the Firefox icon for the Windows 7 thing, that's a pain. These index tabs were in plain sight. No matter what I try in Firefox 8, they won't appear.
the tabs were move to above the navigation bar, where the menu bar was. They hardly look like tabs there, but you may have turned off the tabs bars. View (Alt>V) > Toolbars > Tabs Bar
I think the solutions would really prefer though would be
You can make '''Firefox 8.0''' look like Firefox 3.6.*, see numbered '''items 1-10''' in the following topic [http://dmcritchie.mvps.org/firefox/firefox-problems.htm#fx4interface Fix Firefox 4.0 toolbar user interface, problems (Make Firefox 4.0 thru 8.0, look like 3.6)]. ''Whether or not you make changes, you should be aware of what has changed and what you have to do to use changed or missing features.''
* http://dmcritchie.mvps.org/firefox/firefox-problems.htm#fx4interface
<p>There is a lot more beyond those first 10 steps listed, if you want to make Firefox more functional.</p>
<p><small>Please mark "Solved" one answer that will best help others with a similar problem -- hope this was it.</small></p> -
Materialized view is not taking the Index
Hi Friends,
I have a materialized View for my Sales Details. I created a view on the Materialized View and using in my report. The Performance is very slow. After little research,
I found that the indexes are not used when the MV is accessed in the SQL. I tried explain plan for a very simple query on the view to prove that.
Here is my simple query.
select month_num ,sum(qty) from v_mv_sales_table
where month_num=201301
by month_num;
When I checked for count() it is considering the index and performance is good.
select month_num ,count(*) from v_mv_sales_table
where month_num=201301
by month_num;
Then I tried using hint for forcing the index. That time also it is working only for count not for SUM.
select /* +INDEX(tr, IDX2_MV_SALES_TABLE) */ month_num,count(*)
v_mv_sales_table tr
month_num=201301
by month_num;
Is there any way , I can force my sql to use the index on the month_num column?.
I have a question about using hint to force the sql.
My Question is I have indexes on my Materialized View. But I am using the hint on my view based on the Mat View.
Will it be okay to use like that?.
You may ask why I am using View instead of using Mat View directly. At later stage we are planning to apply some user level restrictions to the data
by linking the view to some other tables/view. So I am forced to use the View instead of MV.
Any ideas or suggestions will be good
Thanks in advance,
Vimal....here's an example
SQL> set autot off
SQL>
SQL> create table t
2 (dt date)
3 /
Table created.
SQL>
SQL> insert into t
2 select trunc (sysdate) + rownum
3 from all_objects
4 /
25707 rows created.
SQL>
SQL>
SQL> create index tidx on t (trunc (dt))
2 /
Index created.
SQL>
SQL> begin
2 dbms_stats.gather_table_stats (user, 'T');
3 end;
4 /
PL/SQL procedure successfully completed.
SQL>
SQL>
SQL> set autot trace expl
SQL> select *
2 from t
3 where dt = to_date ('24-07-2009', 'dd-mm-yyyy')
4 /
Execution Plan
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=6 Card=1 Bytes=8)
1 0 TABLE ACCESS (FULL) OF 'T' (Cost=6 Card=1 Bytes=8)
SQL>
SQL> select *
2 from t
3 where dt = trunc (sysdate)
4 /
Execution Plan
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=6 Card=1 Bytes=8)
1 0 TABLE ACCESS (FULL) OF 'T' (Cost=6 Card=1 Bytes=8)
SQL>
SQL> select *
2 from t
3 where trunc (dt) = trunc (sysdate)
4 /
Execution Plan
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Bytes=8)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=2 Card=1 Bytes=
8)
2 1 INDEX (RANGE SCAN) OF 'TIDX' (NON-UNIQUE) (Cost=1 Card=1
SQL>
SQL>
SQL> drop table t
2 /
Table dropped.
SQL> -
Hello people from the forum,
I have made a beatifull website, and upload it with filezilla to a server, and also I upload it to a local map.
Everthing was going very well, but i don't get my website on the internet, because one page will not be uploaded (again) and that is my first page, the home page (and for as far I understand also my index.html)
I web telling all the time: can not publish this page because there is an unknown error.
The strange thing is, that I can see, even on the local map as on the online server, that my home page is still there!
the size of the index.html is 303 kb (and when I open it, it is empty, but maybe thats normal?)
And yes, I did a lot of stupide things, by accident I deleted a lot of files and documents on my computer :-((
I reinstall Iweb (3.04) but the error is still there.
So, do I have to reinstall OS lion totaly? But them I i'm afraid that I lost all my webpages (even they are on the server now)
(can I put all on a USB stick, and make later again a new homepage?)
I hope one of you recognize this problem and have a more simpeler solution than my story here above.
Sorry for my english, I hope you understand.Hello Wyodor,
thanks for your reply on my question.
You are wright the index file is 308 KB.
And offcourse that I delete my files and documents is not an Iweb problem, but because my English is not that good, that I thougt I try to tell the whole story, and hope people can read trough the lines.
Why I didn't use Iweb to upload to the server is because the server company recommanded to use file zilla. And I am a nobody with this kind of things, so I did exactly what they tell me to do.
No, i did not make a backup of Iweb and the domain file, but reading other messages on this forum showed me that I can maybe try to make empty the cashfile (?)
and otherwise I have bad luck and have to make my website again. Offcourse I have a lot of examples from all the pages (copied from the local map to my USB stick) and my website is aprox. 15 Mb, (not so big) so it will cost me a few nigths.
But I'm still hoping on a miracle :-)
greetings Lasource
Maybe you are looking for
-
How To Set Up Airport Express To Use With iPad2 For My Needs ?
On the advice of several people in a audio forum, I just purchase an Apple Airport Express. I will be using it with my new iPad 2. What I need to accomplish is using my iPad2 to stream subscription based music from the internet to the Airport Express
-
So i bought music on a different computer but oAn the same itunes account and on the same ipod and when i get on my original computer and i sync my ipod my purchases that were on the other computer didnt show up. How can i put them on my computer/ipo
-
Can't see the podcast picture and episodes on iTunes
Hi, My podcast feed was added to iTunes store about a week ago. It was working well for a couple of days, and now, I can't see my episodes and my podcast picture on iTunes catalogue. I was wondering what might be the problem? Here's the XML file crea
-
Delete record works too well !
I've done this before successfully several times but now something strange is happening. I've stared at this for a couple of hours now and don't see what I'm missing, it must be something blindingly obvious. I created a page showing all records in a
-
my ipod charges(i think) but when i take out the charger and try to play music it cant work. but with the charger,it can play music.