Oracle SQL Connector for HDFS - Where is the filter applied?

I have just managed to get the Oracle SQL Connector for HDFS (YEAH!)
working on a Linux 2VM cluster and was curious where the filter in the where clause is being applied to limit the data.
i.e. is it being pushed clear down into hadoop or is hadoop pulling all the data and then then it's being filtered or does all the data get buffered into the database or what?
Thanks
Mark

Along the same lines of this question, does anybody know a way to connect from Oracle to Hadoop/HDFS in such a way as to push the query filter down into the Hadoop query?
I really don't want to pull ALL my hdfs data into the Oracle buffer cache for processing!
Thanks
Mark

Similar Messages

How to use Oracle Direct Connector for HDFS(DCH)?

hi all,
I am trying to use Oracle database to access HDFS through DCH. Now I installed Oracle database 11.2, hadoop-0.20.2 and DCH on the same linux machine.
I also configured hdfs_stream properly and created Oracle directory HDFS_BIN_PATH according to README.txt file within DCH.
Now I encountered a problem, how I create a external table which access data residing on HDFS?
Can anyone help me ?
Thanks in advance.
Chopsun
Edited by: user1745344 on 2012-5-21 下午11:30

I figure out the way which access HDFS data from Oracle External Table through DCH as below, but still face some problems
Step 1
Created Oracle directories as following:
CREATE OR REPLACE DIRECTORY hdfs_bin_path AS '/home/oracle/orahdfs/bin';
CREATE OR REPLACE DIRECTORY data_dir AS '/user/hadoop';
where /user/hadoop is HDFS directory in which data file test.txt located.
Step 2
Created Oracle external table as following:
create table xqs_ext_test(col1 varchar2(100))
organization external (
TYPE oracle_loader
DEFAULT DIRECTORY data_dir
ACCESS PARAMETERS (
records delimited by newline
preprocessor hdfs_bin_path:'hdfs_stream'
badfile hdfs_bin_path:'empxt%a_%p.bad'
logfile hdfs_bin_path:'empxt%a_%p.log'
fields terminated by ','
LOCATION ('test.txt'))
reject limit unlimited
parallel 2;
when run SQL statement : select * from xqs_ext_test; the following error was prompted:
SELECT * FROM xqs_ext_test
ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04040: file test.txt in DATA_DIR not found
who can help me out with this problem?
Best Regards
Chopsun
Edited by: user1745344 on 2012-5-22 上午1:02

Oracle SQL Developer for OSX not able to compile/execute/handle packages?

Hi,
Have been stuck with this for quite some mins now,
When I execute this simple -valid- create package statement in Oracle SQL Developer:
http://www.pastebin.ca/757859
It shows it's not compiled and has errors, when I check the Packages part in Oracle SQL Developer I see this:
http://www.pastebin.ca/757860
The ; after end has been removed by the application. In fact this means I can't do anything with packages in Oracle SQL Developer for OSX?
Is this a known bug?
Or am I doing something wrong?
Any resolving response to this matter would be really appreciated.

Yeah, that bug is in there since stoneage, and is really a no-brainer to fix. Until they don't, we'd be losing time recompiling.
FWIW, everything works fine using the procedure editor for creating, editing and compiling.
Regards,
K.

How do I find out what the charges on my credit card are for? Where is the iTunes account to log into and check?

How do I find out what the charges are on my credit card are for. Where is the iTunes account to sign in and check?

Launch iTunes then select iTunes Store from the source list left side of the window.
Click Sign In on the right side of the window just above Quick Links.
Now click your account name.
In the next window click See All to the right of Purchase History.

BUG in Oracle SQL Developer 3.0.04 on the "generating DLL" with comments?

I'm newbie on oracle, but I think that I found out a bug in Oracle SQL Developer version 3.0.04 on the "generating DLL" tool using "comments".
I will describe the steps that I gave:
I created a view, but after I test it I had to change my “where” condition, so I comment the old code and then I wrote the new “where” condition below. After I done that I tried to look at the sql code of my view using “generating DLL” tool, but oracle sql developer only shown me half of the code, a lot of code were missing. Then I began with some test trying to understand what happen and I notice that if I put an invalid sql code in my comment the generating DLL start working with no problems, for example(pseudo-code):
(COMMENT WITH VALID SQL CODE the "Generating SQL" don't work:)
CREATE OR REPLACE VIEW <user>.<view_name> ( <column1>,<column2> )
AS
SELECT column1, column2
FROM table1
INNER JOIN
(SELECT
FROM table2
INNER JOIN .....
INNER JOIN ....
--where time_stamp = (select max(time_stamp) from .....)
WHERE time_stamp >= TRUNC(sysdate)
) t1 ON t1.ID = ....
AND ..... >= TRUNC(sysdate)
ORDER BY ....
Generating DLL returns this(when the error occurs):
CREATE OR REPLACE VIEW <user>.<view_name> ( <column1>,<column2> )
AS
(COMMENT WITH VALID SQL CODE the "Generating SQL" work with no problems:)
CREATE OR REPLACE VIEW <user>.<view_name> ( <column1>,<column2> )
AS
SELECT column1, column2
FROM table1
INNER JOIN
(SELECT
FROM table2
INNER JOIN .....
INNER JOIN ....
--where
WHERE time_stamp >= TRUNC(sysdate)
) t1 ON t1.ID = ....
AND ..... >= TRUNC(sysdate)
ORDER BY ....
I believe that "Generating DLL" tool have some problem with the comments, I also used /*...*/ to comment but the problem is still active.
I notice as well that if I started to add some more comments along the code, the conditions migth change, so I think the problem is related with "comments" code.
Would you mind telling me if this is a real bug or if I'm doing anything wrong.
Thank you in advance,
Rodrigo Campos
Edited by: 894886 on 3/Nov/2011 5:29

Hi Rodrigo,
Thank you for reporting this. The only bug I see currently logged on a comment affecting the generated View DDL involves ending the last line of the definition with a comment, which treats the ending semi-colon (even if on a different line) as part of the comment. That is actually related to a low-priority bug against an Oracle database API.
Unfortunately, your pseudo-code is a bit complex. Trying a few quick, simpler tests against the standard HR schema did not reproduce the issue. I tried INNER JOIN, and nested SELECTs. It would help greatly if you could provide a test case compilable against one of the standard schema, like HR or SCOTT.
Regards,
Gary
SQL Developer Team

Oracle SQL query for getting specific special characters from a table

Hi all,
This is my table
Table Name- Table1
S.no    Name
1          aaaaaaaa
2          a1234sgjghb
3          a@3$%jkhkjn
4          abcd-dfghjik
5          bbvxzckvbzxcv&^%#
6          ashgweqfg/gfjwgefj////
7          sdsaf$([]:'
8          <-fdsjgbdfsg
9           dfgfdgfd"uodf
10         aaaa bbbbz#$
11         cccc dddd-/mnm
The output has to be
S.no    Name
3          a@3$%jkhkjn
5          bbvxzckvbzxcv&^%#
7          sdsaf$([]:'
8          <-fdsjgbdfsg
10         aaaa bbbbz#$
It has to return "Name" column which is having special characters,whereas some special chars like -, / ," and space are acceptable.
The Oracle query has to print columns having special characters excluding -,/," and space
Can anyone help me to get a SQL query for the above.
Thanks in advance.

You can achieve it in multiple ways. Here are few.
SQL> with t
2 as
3 (
4 select 1 id, 'aaaaaaaa' name from dual union all
5 select 2 id, 'a1234sgjghb' name from dual union all
6 select 3 id, 'a@3$%jkhkjn' name from dual union all
7 select 4 id, 'abcd-dfghjik' name from dual union all
8 select 5 id, 'bbvxzckvbzxcv&^%#' name from dual union all
9 select 6 id, 'ashgweqfg/gfjwgefj////' name from dual union all
10 select 7 id, 'sdsaf$([]:''' name from dual union all
11 select 8 id, '<-fdsjgbdfsg' name from dual union all
12 select 9 id, 'dfgfdgfd"uodf' name from dual union all
13 select 10 id, 'aaaa bbbbz#$' name from dual union all
14 select 11 id, 'cccc dddd-/mnm' name from dual
15 )
16 select *
17    from t
18   where regexp_like(translate(name,'a-/" ','a'), '[^[:alnum:]]');
        ID NAME
         3 a@3$%jkhkjn
         5 bbvxzckvbzxcv&^%#
         7 sdsaf$([]:'
         8 <-fdsjgbdfsg
        10 aaaa bbbbz#$
SQL> with t
2 as
3 (
4 select 1 id, 'aaaaaaaa' name from dual union all
5 select 2 id, 'a1234sgjghb' name from dual union all
6 select 3 id, 'a@3$%jkhkjn' name from dual union all
7 select 4 id, 'abcd-dfghjik' name from dual union all
8 select 5 id, 'bbvxzckvbzxcv&^%#' name from dual union all
9 select 6 id, 'ashgweqfg/gfjwgefj////' name from dual union all
10 select 7 id, 'sdsaf$([]:''' name from dual union all
11 select 8 id, '<-fdsjgbdfsg' name from dual union all
12 select 9 id, 'dfgfdgfd"uodf' name from dual union all
13 select 10 id, 'aaaa bbbbz#$' name from dual union all
14 select 11 id, 'cccc dddd-/mnm' name from dual
15 )
16 select *
17    from t
18   where translate
19         (
20            lower(translate(name,'a-/" ','a'))
21          , '.0123456789abcdefghijklmnopqrstuvwxyz'
22          , '.'
23         ) is not null;
        ID NAME
         3 a@3$%jkhkjn
         5 bbvxzckvbzxcv&^%#
         7 sdsaf$([]:'
         8 <-fdsjgbdfsg
        10 aaaa bbbbz#$
SQL>

Oracle (SQL) support for Temporal / time period based data

Beyond standard SQL (DATE type etc. and <,>, LEAST, GREATEST, etc.) does Oracle offer anything to help handle time period based data? E.g. overlapping periods, joins when gaps in time periods from 2+ tables, period contiguity checks, etc. I've developed various SQL query techniques, but they're all rather cumbersome using standard SQL.
Oracle Spatial offers geo-location support - is there anything similar for temporal query support?
Any good books / docs on the subject?

Check out Workspace manager for limited out of the box temporal support.
You may also want to have a read through this series of articles about temporal database systems. They aren't Oracle specific, but may help you with your temporal system design.

Download Oracle SQL Plus for Queries

Hello everyone,
Could any one let me know whether I can download free version of sql plus in order to build JDBC programs and execute SQL Queries? If so, could any one forward me the url where I can download the free version of sql plus?
Thanks,
Prathima

3. then the next instruction is to set the library loading path in the environment.
Could any one tell me how to proceed with step3 without running the exe file.Well, the Step 3 is pretty clear, isn't it ?
3. Set the library loading path in your environment to the directory in Step 2 ("instantclient"). On
many UNIX platforms, LD_LIBRARY_PATH is the appropriate environment variable. On Windows,
PATH should be used.=================================================================
If you are on Windows, add the full path till the directory where you unzipped the "InstantClient" in your PATH variable.
1) Right click on "My Computer", which is either on the desktop or "Start" -> "My Computer" (depending on your version of Windows)
2) Click on "Advanced" tab and then click on "Environment variables".
3) Select "Path", click on "Edit" button, add the full path till the directory, keep clicking on "OK" till you exit.
=================================================================
If you are using a Unix flavor or Linux distro, you will have to set the "LD_LIBRARY_PATH" variable in your .profile.
In Linux, the .bash_profile is in your HOME directory (echo $HOME once you log in). Append something like the following to it:
LD_LIBRARY_PATH=<full_path_of_the_directory_where_instant_client_has_been_unzipped>
export LD_LIBRARY_PATH
Same goes for the ".profile" of Unix boxes.
Thereafter, run the .profile/.bash_profile in the current shell or log out and log in again.
$ . ~/.bash_profile
HTH
isotope
Just noticed your mention of "exe file", which hints at your OS being Windoze. You may disregard the Unix/Linux part.
Message was edited by:
isotope

ORACLE SQL QUERIES FOR SAP

Hello All,
Can any body give me the total SQL queries which will be used in SAP.
Thanks&Regards,
Praveen Kondabala

Hi,
If you do need that kind of information, then it is easier that you do not use any.
> Like Oracle DBA
you only need to read the documentation about
1) brtools
2) dbacockpit (I assume you have a fairly recent SAP version)
you can find the information about this two "things" in
SAP on Oracle

ADOBE FORM for SAP - where is the documentation

Hello Everybody,
I started to work with adobe form for two weeks now.
And i am wondering something since two days where is the documentation ?
On adobe forums, we find a quick FAQ.
On SCN there is some documentation.
But where is the official or/and most complete documentation ?
Is the full documentation given when you buy the adobe product or is there a full on line documentation ?
Best Regards
Abdou

Here is documentation for formcalc
http://help.adobe.com/en_US/livecycle/es/FormCalc.pdf
Here is documentation for Scripting (JS)
http://help.adobe.com/en_US/livecycle/es/lcdesigner_scripting_reference.pdf
Here is the documentation on the whole package
http://help.adobe.com/en_US/livecycle/9.0/designerHelp/index.htm?content=000403.html
Here is some additional practice to know how to use LiveCycle
http://www.youtube.com/channel/UC6E8WlISDmgiyyq5ZWk-nVw

How can I install Oracle Database 10g for Solaries (SPARC) from the console

Dear Forum Members,
In my office, I have to installed Oracle Database 10g for Solaries (SPARC). But I have to do it without DISPLAY Monitor.Is it possible install it by remote login to this server using response file (silent mode) or something like that?
If yes. Then How?
If anyone have the exact solution, then I need your feedback. I shall wait for your reply.
Thanks
Aungshuman Paul

There are 2 possible ways to accomplish this.
First,
Silent installation
http://www.informit.com/articles/article.asp?p=174771&rl=1
Second, (cut/paste from other site)
How to install Oracle software remotely?
Remote Software Installation Steps: (For Solaris only)
If you want to install Oracle Software remotely, you should perform the following steps. These steps are applicable only if your source and target machine are running Unix.
For example, you can install Oracle Software from your home from Washington, DC to a target source in California.
1. Pick your source server or machine for remote installation.
2. Check that your CD is in your source CD-ROM drive.
3. On the target machine, find your target machine name with the output of the /usr/bin/hostname
4. On the source machine, login as a user.
5. On the source machine, enable client access: % /usr/openwin/bin/xhost + target-machine-name
6. Become root user by typing: su (dont use -)
7. Check that Volume Manger is running. # ps ef |grep vold (if you see an entry that contains /usr/sbin/vold, Volume Manager is running. Then skip to Step 10.
8. If not then do the following: # mkdir p /cdrom/your-cd-file-name
9. # mount F hsfs r cdrom-device /cdrom/your-cd-file-name
10. Add the following line to your /etc/dfs/dfstab file: # share F nfs o ro /cdrom/your-cd-file-name
11. Verify whether your source machine is an NFS server: # ps ef | grep nfsd
12. If you see an entry that contains /use/lib/nfs/nfsd a 16, then nfsd is running and skip to Step 16.
13. If nfsd is running, then type: # /usr/sbin/shareall
14. If nfsd is not running, then start nfsd by typing: # /etc/init.d/nfs.server start
15. Verify whether your source machine is an NFS server again by typing: # ps ef | grep nfsd
16. Make sure your source machine is exporting your product directory by typing: # /usr/sbin/dfshares
17. Now, log in to the target machine by type: # rlogin target-machine-name l user (not root)
18. Then log in as the root user by typing: # su
19. Go to the source machine by typing: # cd /net/source-machine/cdrom/your-cd-file-name ,then Skip to 24.
20. If you cannot change to that directory in Step 19 and you do not have an auto-mounter on your network, then create amount point by typing the following commands.
21. # mkdir /remote_products
22. # /usr/sbin/mount F nfs r source-machine:/cdrom/your-cd-file-name /remote_products
23. # cd /remote_products
24. Redirect the target machine display to source machine by typing: # DISPLAY=source-machine:0; export DISPLAY (if you use a Bourne or Korn shell).
25. Start the Web Start Installer by typing: # ./installer (or whatever the installer name program is).

XML for Analysis - where's the 7.0 docs?

Trying to install and run 7.01 w XML for Analysis Services, but the doc included with Deployment Services is based on 6.6 and 6.5 - files, locations, etc. are all changed.Anyone know where the real 7.0.X documentation/installation instructions are?

Got to that documentation - it's the 6.6/6.5 config information.What was good news is that most of the config had already been done. What was missing was the URL to the XMLA service on EDS which is:http://<host>:8080/xmla/EssbaseXmlForAnalysisWhich I got from tech support.Now I'm up and running. Works fine (and quickly) w the Microsoft XML/A sample application.

Where did the filter ' diffuse glow' went to ? And how to install it again in PS 2014 ? In PS 2014 is de filter 'gloed onscherp' niet meer te vinden. Iemand en oplossing aub ?

Now I installed PS 2014, I noticed the filter 'diffuse glow' is not there anymore ...
How do I install this one ?
Ik heb zopas PS 2014 geïnstalleerd en vastgesteld dat de filter 'gloed onscherp' niet meer in de lijst staat.
Iemand enig idee hoe ik die terug erbij kan halen aub ?
bedankt,
Rik

Chris, I'm using this filter since it was created.
I know the place where to find it.
in these 2 screenshots you can see that some of the filters are missing ...
gloed onscherp is the one I mentioned.
My question is how to get this one back ?

CacheEventFilter: Where does the filter part apply?

Coherence 3.3.1/389
     .Net API 3.3.2
     It seems, that the filter part of the CacheEventFilter object applies on the client side. I have the object with property named Portfolio on the client side and the object with getPortfolio method on the server side. So when i try to use filter with the method name 'getPortfolio', it does not work and all events come to the client. When i added the same method to the client object all became to work fine.
     Is it correct?
     If this is true, then why don't you do filtering on the server side? It is too expensive to send all events to the client and then filter them.
     Thanks.

Hi Dmitry,
     The AddCacheListener(ICacheListener listener, IFilter filter, bool isLite) method is used to listen to all cache events for which the supplied filter evaluates to true. The isLite flag indicates whether or not the cache events passed to the ICacheListener include the old and new values. An ICacheListener registered using this method must be unregistered by calling the RemoveCacheListener(ICacheListener listener, IFilter filter) method with the same ICacheListener instance and filter.
     The AddCacheListener(ICacheListener listener, Object key, bool isLite) method is used to listen to all cache events that pertain to the cache entry with the specified key. The isLite flag indicates whether or not the cache events passed to the ICacheListener include the old and new values. An ICacheListener registered using this method must be unregistered by calling the RemoveCacheListener(ICacheListener listener, Object key) method with the same ICacheListener instance and key.
     The AddCacheListener(ICacheListener listener) method is used to listen to all cache events. An ICacheListener registered using this method will be passed cache events with both the old and new values and must be unregistered by calling the RemoveCacheListener(ICacheListener listener) method with the same ICacheListener instance.
     Regards,
     Jason

Where are the filter/effect presets saved on a Mac?

I've searched a bit but can't seem to find anything but tutorials on saving your own presets.
On my old Mac I had a few presets which I liked, and so I want to copy them to my new Mac. I have a Time Machine backup and I managed to find my export presets, but need a bit of help on the filter/effect presets.
Thanks in advance!

Good to know, but unfortunately my old computer broke so I can't export the presets, I only have the Time Machine back up.
Surely they get saved somewhere in the program files?

Oracle SQL Connector for HDFS - Where is the filter applied?

Similar Messages

Maybe you are looking for