Big data and database administration

Hi,
I am working as a Oracle DBA. I would like to know what is dba role for Big data & Nosql.
Is it really useful for learning bigdata.
Thanks,

. Are
there any relationship between these two fields?You are comparing cheese with chalk.
how
I can learn more about the data wherehousing?Start with Oracle doc,
Oracle® Database Data Warehousing Guide
http://download.oracle.com/docs/cd/B19306_01/server.102/b14223/toc.htm

Similar Messages

Is the only way to import large amount of data and database objects into a primary database is to shutdown the standby, turn off archive log mode, do the import, then rebuild the standby?

I have a primary database that need to import large amount of data and database objects. 1.) Do I shutdown the standby? 2.) Turn off archive log mode? 3.) Perform the import? 4.) Rebuild the standby? or is there a better way or best practice?

Instead of rebuilding the (whole) standby, you take an incremental (from SCN) backup from the Primary and restore it on the Standby. That way, if, for example
a. Only two out of 12 tablespaces are affected by the import, the incremental backup would effectively be only the blocks changed in those two tablespaces (and some other changes in system and undo) {provided that there are no other changes in the other ten tablespaces}
b. if the size of the import is only 15% of the database, the incremental backup to restore to the standby is small
Hemant K Chitale

Data wherehousing and Database Administration

Dear All
Currently I'm working as a Trainee DBA.I am really interesting to woyrking with Data wherehousing . Are there any relationship between these two fields? how I can learn more about the data wherehousing?

. Are
there any relationship between these two fields?You are comparing cheese with chalk.
how
I can learn more about the data wherehousing?Start with Oracle doc,
Oracle® Database Data Warehousing Guide
http://download.oracle.com/docs/cd/B19306_01/server.102/b14223/toc.htm

What is the difference between Data Warehousing and Big Data?

What is the difference between Big Data and
Data Warehousing? are they the same? similar? if no, then when to use each of them?
Any link to a paper that describes the difference?!

Big Data is a term applied to data sets whose size is beyond the ability of commonly used tools to capture, manage and process the data within a tolerable elapsed time. But Data-warehouse is a collection of data marts representing historical data from different
operations in the company.
It means Big Data is collection of large data in a particular manner but Data-warehouse collect data from different department of a organization. However Data-warehouse require efficient managing technique. Conceptually these are same only at one factor that
they collect large amount of information
For Big Data - You can start researching on HDInsight and for Datawarehouse check MSBI
Sandip Pani http://SQLCommitted.com

What is the best big data solution for interactive queries of rows with up?

0 down vote favorite
We have a simple table such as follows:
| Name | Attribute1 | Attribute2 | Attribute3 | ... | Attribute200 |
| Name1 | Value1 | Value2 | null | ... | Value3 |
| Name2 | null | Value4 | null | ... | Value5 |
| Name3 | Value6 | null | Value7 | ... | null |
| ... |
But there could be up to hundreds of millions of rows/names. The data will be populated every hour or so.
The goal is to get results for interactive queries on the data within a couple of seconds.
Most queries look like:
select count(*) from table
where Attribute1 = Value1 and Attribute3 = Value3 and Attribute113 = Value113;
The where clause contains arbitrary number of attribute name-value pairs.
I'm new in big data and wondering what the best option is in terms of data store (MySQL, HBase, Cassandra, etc) and processing engine (Hadoop, Drill, Storm, etc) for interactive queries like above.

Hi,
As always, the correct answer is "it depends".
- Will there be more reads (queries) or writes (INSERTs)?
- Will there be any UPDATEs?
- Does the use case require any of the ACID guarantees, or would "eventual consistency" be fine?
At first glance, Hadoop (HDFS + MapReduce) doesn't look like a viable option, since you require "interactive queries". Also, if you require any level of ACID guarantees or UPDATE capabilities the best (and arguably only) solution is a RDBMS. Also, keep in mind that Millions of rows is pocket change for modern RDBMSs on average hardware.
On the other hand, if there'll be a lot more queries than inserts, VERY few or no updates at all, and eventual consistency will not be a problem, I'd probably recommend you to test a Key-Value store (such as Oracle NoSQL Database). The idea would be to use (AttributeX,ValueY) as the Key, and a Sorted List of Names that have ValueY for their AttributeX. This way you only do as many reads as attributes you have in the WHERE clause, and then compute the intersection (very easy and fast with sorted lists).
Also, I'd do this computation manually. SQL may be comfortable, but I don't think It's Big Data ready yet (unless you chose the RDBMS way, of course).
I hope it helped,
Joan
Edited by: JPuig on Apr 23, 2013 1:45 AM

Big data Implementation in sql server 2008 r2

Can Some one please explain what is big data and implementation of big data or usage of big data with clear explanation, Screen shots will be more helpfull.
Thanks in Advance.
Tanks,
G.Satish Reddy.

Can Some one please explain what is big data and implementation of big data or usage of big data with clear explanation, Screen shots will be more helpfull.
Unfortunately, the data is to big to fit into a screenshot. :-)
Big data is one the most recent buzzwords in the computer industry. SQL 2008 R2 is not a good tool to work with big data.
Big data is about analysing patters in big amounts of data, often semi-structured or unstructured. Places like Facebook, Google, Amazon spend a lot of time in analysing click logs to see which pages you went to and from, in that way they can predict your
interest.
Much of the Big Data analysis is performed with Hadoop, which is an open source offering that works on a non-relational database. Someone mentioned PDW as Microsoft's offering in the space, but Microsoft works with Hadoop as well, and they also marry
them together.
Anyway, if you relaly want to get a grip what this is all about, listen David DeWitt's excellent keynote from the PASS Summit 2011. You will have to register and become a member of PASS if you aren't already. PASS is free and it is a non-profit organisation
for SQL users world-wide.
http://www.sqlpass.org/summit/2011/live/livestreaming/livestreamingfriday.aspx
Erland Sommarskog, SQL Server MVP, [email protected]

Date difference function that returns minutes between two dates and excludes weekends and holidays

Is there a way to get this to work as a function?
Currently returns error "Select statements included within a function cannot return data to a client"
CREATE FUNCTION [dbo].[WorkDays](@DateFrom datetime,@DateTo datetime)
RETURNS int
AS
BEGIN
--Working time
DECLARE @WTFrom TIME = '8:00AM';
DECLARE @WTTo TIME = '5:00PM';
DECLARE @minCount BIGINT
--Date ranges
IF (DATEDIFF(HOUR, @DateFrom, @DateTo) > 12)
BEGIN
WITH CTE AS
SELECT @DateFrom AS DateVal
UNION ALL
SELECT DATEADD(HOUR, 1, DateVal)
FROM CTE
WHERE DateVal < DATEADD(HOUR, -1,@DateTo)
SELECT DATEDIFF(minute, MIN(CTE.DateVal), MAX(CTE.DateVal))
FROM CTE
WHERE (CAST(CTE.DateVal AS time) > @WTFrom AND CAST(CTE.DateVal AS time) < @WTTo) AND DATEPART(dw, CTE.DateVal) NOT IN (1, 7) AND NOT EXISTS (SELECT * FROM Holiday AS H WHERE H.holiday = CTE.DateVal)
OPTION (MAXRECURSION 0);
END;
ELSE
BEGIN
WITH CTE AS
SELECT @DateFrom AS DateVal
UNION ALL
SELECT DATEADD(MINUTE, 1, DateVal)
FROM CTE
WHERE DateVal < DATEADD(MINUTE, -1,@DateTo)
SELECT DATEDIFF(minute, MIN(CTE.DateVal), MAX(CTE.DateVal))
FROM CTE
WHERE (CAST(CTE.DateVal AS time) > @WTFrom AND CAST(CTE.DateVal AS time) < @WTTo) AND DATEPART(dw, CTE.DateVal) NOT IN (1, 7) AND NOT EXISTS (SELECT * FROM Holiday AS H WHERE H.holiday = CTE.DateVal)
OPTION (MAXRECURSION 0);
END;
END
Thanks for your help.

Please post DDL, so that people do not have to guess what the keys, constraints, Declarative Referential Integrity, data types, etc. in your schema are. Learn how to follow ISO-11179 data element naming conventions and formatting rules (you do not). Temporal
data should use ISO-8601 formats (you do not!). Code should be in Standard SQL as much as possible and not local dialect.
This is minimal polite behavior on SQL forums.
We hate functions in SQL. This is a declarative language and you are using it like 1950's FORTRAN. We hate local variables (more FORTRAN!)
The name of a function has to be either a known, common name, like “sine”, “cosine” etc. Or it is “<verb>_<object>”; you think a noun is a verb! Do you really need BIGINT? Why did you invite garbage data with it? Why do you think that SQL
uses AM/PM? Have you never seen the TIME data type?
Think about “date_val” as a data element name. A date is a unit of temporal measurement on a calendar scale. This would be a “<something>_date” in a valid schema.
>> Is there a way to get this to work as a function? <<
Probably, but why do it wrong?
Build a calendar table with one column for the calendar data and other columns to show whatever your business needs in the way of temporal information. Do not try to calculate holidays in SQL -- Easter alone requires too much math.
The julian_business_nbr is how SQL people do this. Here is the skeleton.
CREATE TABLE Calendar
(cal_date DATE NOT NULL PRIMARY KEY,
julian_business_nbr INTEGER NOT NULL,
Here is how it works:
INSERT INTO Calendar
VALUES ('2007-04-05', 42),
('2007-04-06', 43), -- Good Friday
('2007-04-07', 43),
('2007-04-08', 43), -- Easter Sunday
('2007-04-09', 44),
('2007-04-10', 45); --Tuesday
To compute the business days from Thursday of this sample week to next Tuesday:
SELECT (C2.julian_business_nbr - C1.julian_business_nbr)
FROM Calendar AS C1, Calendar AS C2
WHERE C1.cal_date = '2007-04-05',
AND C2.cal_date = '2007-04-10';
See how simple it can be when you stop trying to write FORTRAN and think in sets?
--CELKO-- Books in Celko Series for Morgan-Kaufmann Publishing: Analytics and OLAP in SQL / Data and Databases: Concepts in Practice Data / Measurements and Standards in SQL SQL for Smarties / SQL Programming Style / SQL Puzzles and Answers / Thinking
in Sets / Trees and Hierarchies in SQL

Best strategy to upload a big data file and then to insert in db the content

Hi,
Here's our requirement. We have a web business application developed on JSF 1.2, JE66, WebLogic for application server, and Oracle for the data back end tier. We need to upload big data files (80 to 100 Mb) from a web page, and persist the content in database tables.
What's the best way in terms of performance to implement this use case ? Once the file is uploaded on server a command button is available on the web page to trigger a JSF controller action in order to save data in database.
Actually we plan to keep in memory the content of the http request, and call insert line on each line of the file. But i think it's bad and not scalable.
Is is better to write the file on server's disk and then use multiple threads to send the lines to the database ? How to use multi threading in a JSF managed bean ?
Thanks

In addition, LoadFromFile is overloaded to handle both BLOB and CLOB:
PROCEDURE LOADFROMFILE
Argument Name                  Type                    In/Out Default?
DEST_LOB                       BLOB                    IN/OUT
SRC_LOB                        BINARY FILE LOB         IN
AMOUNT                         NUMBER(38)              IN
DEST_OFFSET                    NUMBER(38)              IN     DEFAULT
SRC_OFFSET                     NUMBER(38)              IN     DEFAULT
<BR>
PROCEDURE LOADFROMFILE
Argument Name                  Type                    In/Out Default?
DEST_LOB                       CLOB                    IN/OUT
SRC_LOB                        BINARY FILE LOB         IN
AMOUNT                         NUMBER(38)              IN
DEST_OFFSET                    NUMBER(38)              IN     DEFAULT
SRC_OFFSET                     NUMBER(38)              IN     DEFAULT

Upcoming Lumira Webinar June 11th on the topic of "Big Data Visualization and Custom Extensions"

The next webinar in the Lumira series is coming up this week on Wednesday, June 11th, 10:00 AM - 11:00 AM Pacific Standard Time. The day is almost here and if you've already registered, thank you!
If not, please click here to register.
Speaker Profile:
The topic is presented by the Jay Thoden van Velzen, Program Director Global HANA Services/Big Data Services Center of Excellence at SAP!
Jay has been working in Analytics/Business Intelligence since it was called Decision Support Systems in the late 90s. Currently he is focused on Big Data solutions and how to make the various components of such a solution run smoothly integrated together using the SAP HANA Platform.
Abstract:
Big Data analysis poses unique and new challenges to data visualization, compared to more traditional analytics. Such analysis often includes frequency counts, analysis of relationships in a network, and elements of statistical and predictive modeling. In many cases, traditional visualization techniques of bar- and column charts, pie charts and line graphs are not the most appropriate. We have to avoid the “beautiful hairball” and make it easy for end users to absorb the information through clever use of filtering, transparency and interactivity. We will likely also need to provide more context to go with the visualization than we have been used to in traditional analytics. Moreover, in case of forecasts you need to include any confidence intervals in order not to mislead.
This means we need more chart types, and often the chart types you need may not exist, nor could the need for such chart types necessarily be foreseen. However, Lumira allows us to design and code our own D3.js visualizations and integrate it into Lumira while providing all the data access methods – including SAP HANA – that it provides out of the box. This means we can develop our visualizations to share the outcomes of Big Data analysis to exactly how we feel it should be presented. During the webinar we will show a number of examples, and specifically the integration of forecasts coming from R into Lumira through a Lumira custom extension.
We really hope to see you there!
Cheers!
Customer Experience Group

Congrats to Joao and Alex!
Microsoft Azure Technical Guru - May 2014
João Sousa
Microsoft Azure - Remote Debbuging How To?
GO: "Clever. Well Explained and written. Thanks! You absolutely deserve the GOLD medal."
Ed Price: "Fantastic topic and great use of images!"
Alex Mang
The Move to the New Azure SQL Database Tiers
Ed Price: "Great depth and descriptions! Very timely topic! Lots of collaboration on this article from community members!"
GO: "great article but images are missing"
Alex Mang
Separating Insights Data In Visual Studio Online
Application Insights For Production And Staging Cloud Services
Ed Price: "Good descriptions and clarity!"
GO: "great article but images are missing"
Ed Price, Power BI & SQL Server Customer Program Manager (Blog,
Small Basic,
Wiki Ninjas,
Wiki)
Answer an interesting question?
Create a wiki article about it!

I am using the big date calendar template and when I submit it to apple for printing I lose the name of two months. These names are not text boxes. I see the names when I send it in but something happens during the transmission to apple. It was suggested

I am using the big date calendar template in iPhoto. I am on Lion 10.7.2, macbook air. The names of the months are on each calendar page but something happens when I send the data to Apple. The names are part of the template. They are not text boxes. I lose two names on the calendar after it is sent to Apple. Apple suggested I make a pdf file of my calendar before sending it in and check to make sure every name shows. I did this with a calendar I just sent in. The calendar was correct. All names of the months were showing. After sending the data two month names disappeard because when it arrived by mail, it was incorrect. Apple looked at my calendar via a pdf file and it was incorrect. This is second time this has happened. I called Apple and they had me delete several folders in the Library folder, some preferences and do a complete reinstall of iPhoto. I have not yet remade the defective calendar. I am wondering if anyone else has had this problem?
kathy

Control-click on the background of the view all pages window and select "Preview Calendar" from the contextual menu.
You can also save the pdf as a file to compare to the printed calendar. If the two names are visible in the pdf file then the printed copy should show them. Contact Apple for a refund. Apple Print Products - Apple Store (U.S.)

Help on export sybase iq tables with data and import in another database ?

Help on export Sybase iq 16 tables with data and import into another database ?

Hi Nilesh,
If you have table/index create commands (DDLs), you can create them in Developper and import data using one of methods below
Extract/ Load table
Insert location method : require IQ servers to be entered in interfaces file
Backup/Restore : copy entire database content
If you have not the DDLs, you can generate them using IQ cockpit or SCC.
http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01773.1604/doc/html/san1288042631955.html
http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01840.1604/doc/html/san1281564927196.html
Regards,
Tayeb.

[webdynpro] How to get the data from database and store in Excel sheet

Hi All-
I am developing an application in Webdynpro and I need to provide a URL ( link ) which if clicked , need to collect the data from Database ( SQL Server ) and puts in an Excel Sheet corresponding fields and opens the sheet.....
Please look into this issue and help me out......
Regards,
Cris

Hi Cris,
Add-on to wat santosh has pointed to:
Exporting table data to MS-Excel Sheet(enhanced Web Dynpro Binary Cache)
(Or) If you have implemented your logic to get Database records below Blog should guide you in opening an excel with ur records.
Exporting table data to MS-Excel Sheet(enhanced Web Dynpro Binary Cache)
Regards,
N.

Difference between "Database Variant To Data" and "Variant To Data"

Can anyone tell me the differnce between the 2 VI's "Database Variant To Data" and "Variant To Data"?
I am using the database connectivity toolset and making a query.
I have tried using both, but the give me the same result.
Attachments:
Variant to Data.JPG ‏45 KB

You won't see a difference between these two functions with just a string or other simple data types. Where you'll see a difference is with clusters and more complex data types that come from the database. The variants returned by databases can be slightly different than the standard variant. That is where the Database Variant To Data is needed. Otherwise, if you are just reading string information, either function will work fine.
Crystal

Script to find locks in Database with Date and time.

Hi all,
I used to have a script which i have lost now due to a recent laptop crash. This script used to find locks in the database.. along with the date and time since when it is locking the session.
It was quite a comprehisive one. I think i found it on this forum only, but not able to find it now. Can someone please help me find one such comprehensive script.
Currently I am using this script.. which is also good.. but does not have the date and time stamp on it. Is there something better.?
SET LINESIZE 165
SET PAGESIZE 66
COLUMN oracle_user     FORMAT a15      HEADING 'Oracle User'
COLUMN usercode        FORMAT a12      HEADING 'SID/Serial#'
COLUMN os_user         FORMAT a10      HEADING 'O/S User'
COLUMN program         FORMAT a25      HEADING 'Program'
COLUMN mode_held       FORMAT a15      HEADING 'Mode Held'
COLUMN mode_requested FORMAT a15      HEADING 'Mode Requested'
COLUMN lock_type       FORMAT a15      HEADING 'Lock Type'
COLUMN object_name     FORMAT a30      HEADING 'Object Name'
COLUMN lock_time_min   FORMAT 999,999 HEADING 'Lock Time (min)'
SELECT
s.username                                 oracle_user
+, l.sid || '/' || s.serial# usercode+
+, s.osuser os_user+
+, s.program program+
+, DECODE(l.lmode,+
+1, NULL,+
+2, 'Row Share',+
+3, 'Row Exclusive',+
+4, 'Share',+
+5, 'Share Row Exclusive',+
+6, 'Exclusive', 'None') mode_held+
+, DECODE(l.request,+
+1, NULL,+
+2, 'Row Share',+
+3, 'Row Exclusive',+
+4, 'Share',+
+5, 'Share Row Exclusive',+
+6, 'Exclusive', 'None') mode_requested+
+, DECODE(l.type,+
+'MR', 'Media Recovery',+
+'RT', 'Redo Thread',+
+'UN', 'User Name',+
+'TX', 'Transaction',+
+'TM', 'DML',+
+'UL', 'PL/SQL User Lock',+
+'DX', 'Distributed Xaction',+
+'CF', 'Control File',+
+'IS', 'Instance State',+
+'FS', 'File Set',+
+'IR', 'Instance Recovery',+
+'ST', 'Disk Space Transaction',+
+'TS', 'Temp Segment',+
+'IV', 'Library Cache Invalidation',+
+'LS', 'Log Start or Log Switch',+
+'RW', 'Row Wait',+
+'SQ', 'Sequence Number',+
+'TE', 'Extend Table',+
+'TT', 'Temp Table',+
l.type)                                 lock_type
+, o.owner || '.' || o.object_name+
+|| ' - (' || o.object_type || ')' object_name+
+, ROUND(l.ctime/60, 2) lock_time_min+
FROM
v$session     s
+, v$lock l+
+, dba_objects o+
+, dba_tables t+
WHERE
l.id1            = o.object_id
AND s.sid            = l.sid
AND o.owner          = t.owner
AND o.object_name    = t.table_name
AND o.owner          <> 'SYS'
AND l.type           = 'TM'
ORDER BY
+1+
+/+

What Brian said is true so are your sure your query was not just reporting the start time of the current transaction from v$transaction.start_time, or the start time of the session: v$session.logon_time, or perhaps the time since the last database call (v$session.last_call_et) which for an active session would be the time the current SQL statement has been running?
HTH -- Mark D Powell --

Drag and drop to view data from database

hi,
totally new to dreamweaver cs4, had some real helpful pointers already on there forums so thanks.
i was wondering, how can you drag and drop in dreamweaver cs4 to display contents of a mysql database? i have worked a bit with .net vwd in past and you dragged over say a list view to display data and really just want to be able to do the same in dreamweaver cs4 but not sure what it would come under in the php menu bar or data option in menu bar?
can anyone point me in right direction?
many thanks
andy

deansy55 wrote:
i was wondering, how can you drag and drop in dreamweaver cs4 to display contents of a mysql database?
Read the Help pages starting here: http://help.adobe.com/en_US/Dreamweaver/10.0_Using/WScbb6b82af5544594822510a94ae8d65-78d3a .html.
There's also a tutorial here: http://www.adobe.com/devnet/dreamweaver/articles/first_dynamic_site_pt1.html.
Basically, you create a recordset, and then drag the results from the Bindings panel.

Big data and database administration

Similar Messages

Maybe you are looking for