Storing 6 millions values in SQL server columns

Hi,
How many values (size) I can store in single SQL server column.
My Scenario,
Column1 Column2 Column3
1 ABC 1,2,3,........(6 millions values)
2 CDE 1,2,3,.......(6 millions values)
and problem is that I can't store them in rows. I need to store them in single column with comma separated.
What would be the best way to store them and retrieve them faster. I am thinking about converting it to byte[] or something but not sure how much time it will take to convert to byte and then again to text.

and problem is that I can't store them in rows. I need to store them in single column with comma separated.
No you don't! Unless, that is, you really like to hurt yourself. It's a basic idea that in a relational database that each cell holds an atomic value, and this what relational datbases are optimised for.
That said, if all you want to do is to store a bunch of values that you will never look at in the database, but the database will act as an unintelligent data store you will only look at the values outside the database, then it
could make sense. However, a warning: just because you think that there is no requirement today, don't be surprised that you are sooner or later are asked questions like: "which the highest value for ABC?", "Which value is the most
frequent across ABCs and CDEs?". Questions that are very simple to answer is you have data in a properly designed data model. But extremely to painful to answer if your all values in a single column.
But if you take this, path, yes, storing values in a varbinary(MAX) with four bytes (or whatever that is needed) per value would be the best thing. It takes up less space than a string, and with fixed length it's easier to find value 198713 if this would
be nedded. You seem to be worried about conversion from byte to text. Why is not clear to me, since number are usally born binary. But obviously, I have no idea where you get these values from or how you would use them.
My initial recommendation still stands: store each value in a single row.
Erland Sommarskog, SQL Server MVP, [email protected]

Similar Messages

[[DataDirect][ODBC SQL Server Wire Protocol driver][Microsoft SQL Server]Column name or number of supplied values does not match table definition.]

Hii ..
need help on this ..
This what I am doing ..
I am using a DATAEXPORT function to export level0 data from my essbase 11.1.2.2 to Microsoft SQL 2008 tables.
So what I did first I exported the level0 data to a flat file using DATAEXPORT and the created the SQL columns by the same in that order only in my SQL table.
When I run it fails with this error:
ODBC Layer Error: [21S01] ==> [[DataDirect][ODBC SQL Server Wire Protocol driver][Microsoft SQL Server]Column name or number of supplied values does not match table definition.]
[Tue Jul 23 18:26:07 2013]Local/dataexp/dataexp/admin@Native Directory/1209813312/Info(1021014)
ODBC Layer Error: Native Error code [213]
[Tue Jul 23 18:26:07 2013]Local/dataexp/dataexp/admin@Native Directory/1209813312/Error(1012085)
Unable to export data to SQL table [dataexp]. Check the Essbase server log and the system console to determine the cause of the problem.
[Tue Jul 23 18:26:07 2013]Local/dataexp/dataexp/admin@Native Directory/1209813312/Info(1021002)
SQL Connection is Freed
[Tue Jul 23 18:26:07 2013]Local/dataexp/dataexp/admin@Native Directory/1209813312/Warning(1080014)
Transaction [ 0x1c50001( 0x51ee7d66.0x80342 ) ] aborted due to status [1012085].
[Tue Jul 23 18:26:07 2013]Local/dataexp/dataexp/admin@Native Directory/1209813312/Info(1012579)
Total Calc Elapsed Time for [test.csc] : [1.44] seconds
=============================================================
I did a simple test on my Sample.basic application then ..
loaded the calc data to it and then used the below script to export to a flat file
//ESS_LOCALE English_UnitedStates.Latin1@Binary
SET DATAEXPORTOPTIONS
                DataExportLevel "Level0";
                DataExportOverwriteFile ON;
                DataExportColFormat OFF;
                DataExportDimHeader OFF;
FIX(
            "Jan",
            "Sales",
            "Actual"
/*DATAEXPORT "File" "," "/home/hypadmin/samtest.txt";*/
DATAEXPORT "DSN" "Abhitest" "sample" "sa" "welcome1";
ENDFIX
out put as below:
"Sales"
"100-30","California"
"Jan","Actual",145
Now In sql I created only 3 columns with name Jan/Sales/Actual and when I run this script again with comments removed .. I get the same error as what I have got in my first test case with other application ..
but when I create the columns with same name as what its in export
Sales/100-30/Califirnia/Jan/Actual
It created the new rows successfully ..
So with this I think the error which I am getting with my other application might be because of the same column issue .. but then I have created all the columns by looking at the export file only as I did in sample ..
Any idea would be helpful ..
Thanks
Abhishek
I

First make sure you add
DataExportRelationalFile ON;
to your set commands it is missing
I alwats like to also add
DataExportColHeader dimensionName;
so I am sure what dimension is getting put into the columns.
Then count the number of dimensions in your outline (exclude attribute dimensions). You need at least that many columns in your table -1 + the number of members you will be returning as columns in the export
Taking your example Sample basic has 5 dimensions
Measures
Years
Scenario
Product
Market
Since you did not specify a dataexportcolheader it took the dense dimension Scenario as the columns. Your fix statement is limiting that to one member. Doing the math
5 -1 + 1 = 5 columns in your table which is what you found works. Suppose you fixed on bothe Actual and budget in scenario then you would need 6 columns 5 -1 +2

Writing a stored procedure to import SQL Server table data into a Oracle table

Hello,
As a new DBA I have been tasked with writing a stored procedure to import SQL Server table data into an Oracle table. I have been given many suggestions on how to do it from SQL Server but I I just need to write a stored procedure to run it from the Oracle side. Suggestions/guidance on where to start would be greatly appreciated! Thank you!
I started to write it based on what I have but I know this is not correct :/
# Here is the select statement for the data source in SQL Server...
SELECT COMPANY
,CUSTOMER
,TRANS_TYPE
,INVOICE
,TRANS_DATE
,STATUS
,TRAN_AMT
,CREDIT_AMT
,APPLD_AMT
,ADJ_AMT
,TRANS_USER1
,PROCESS_LEVEL
,DESCRIPTION
,DUE_DATE
,OUR_DATE
,OUR_TIME
,PROCESS_FLAG
,ERROR_DESCRIPTION
FROM data_source_table_name
#It loads data into the table in Oracle....
Insert into oracle_destination_table_name (
COMPANY,
CUSTOMER,
TRANS_TYPE,
INVOICE,
TRANS_DATE,
STATUS,
TRANS_AMT,
CREDIT_AMT,
APPLD_AMT,
ADJ_AMT,
TRANS_USER1,
PROCESS_LEVEL,
DESCRIPTION,
DUE_DATE,
OUR_DATE,
OUR_TIME,
PROCESS_FLAG,
ERROR_DESCRIPTION)
END;

CREATE TABLE statements would have been better as MS-SQL and Oracle don't have the same data types.
OUR_DATE, OUR_TIME will (most likely) be ONE column in Oracle.
DATABASE LINK
Personally, I'd just load the data over a database link:
insert into oracle_destination_table_name ( <column list> )
select ... <transform data here>
from data_source_table@mssql_db_link
As far as creating the database link from Oracle to MS-SQL ... that is for somebody else to answer.
(most likely you'll need to use an ODBC driver)
EXTERNAL TABLE
If the data from MS-SQL is in a CSV file, just use and external table.
same concept:
insert into oracle_destination_table_name ( <column list> )
select ... <transform data here>
from data_source_external_table
MK

Encoding character type in sql server column

Hello,
I am having troubles during REPLACE character in sql server column due to fact that, it seems to me, SQL Server does not recognize two "the same" characters i.e., some of them it translates into another encoding system correctly while, some of
them, keeps untouched, as entered. Obviously, according to my understanding, they "look the same shape" but, obviously, belong to different encoding systems. How to change encoding system for some of them?
Namely, this UPDATE statement:
UPDATE TOP (4000000) tblName
SET FirstName = N'Đ' + SUBSTRING(FirstName, 2, 8000)
WHERE (FirstName LIKE N'Ð%')
works fine for my case but, as you see, it replaces just the first occurrence. Could it be possible to expand this pattern to all occurrences?

I'm not sure that I have fully understood what you want to achieve, but it seems that you want to replace occurrances of Latin Captial Letter Edh with Latin Capital Letter D with Stroke, which looks virtually identical in uppercase (but are entirely different
in lowercase). Or vice versa.
Using the glyphs is certainly bound to cause confusion, and I think the best for clarity is to use the nchar function. Or possibly upper in combination with the lowercase glyphs. As for entering the characters, the Character Map utility, which you find in
the folder Accessories/System Tools folder in the Start Menu is useful.
Below is a script that demonstrates the techniques above. You can also see that the characters are considered equal in a accent-insensitive collation. (Well, at least Latin1_General. They are of course unequal in Icelandic and Croatian AI collations.)
CREATE TABLE tblName(FirstName nvarchar(20) NOT NULL)
INSERT tblName(FirstName)
VALUES(upper('eðh'))
SELECT FirstName, lower(FirstName)
FROM tblName
UPDATE dbo.tblName
SET FirstName = REPLACE(FirstName, nchar(208), nchar(272))
WHERE FirstName LIKE N'%' + nchar(208) + '%';
SELECT FirstName, lower(FirstName)
FROM tblName
SELECT CASE WHEN nchar(208) = nchar(272) COLLATE Latin1_General_CI_AI THEN 1 ELSE 0 END AS Equal
go
DROP TABLE tblName
Erland Sommarskog, SQL Server MVP, [email protected]

How to delete number (00) in a SQL Server column ?

how to delete number (00) in a SQL Server column ?
example :
column : Births before   Births
  after
                   199900
        1999
         198200
     1982
                   200400
    2004
help query

You use REPLACE function to selectively replace text inside a string in SQL Server. The REPLACE function is easy to use and very handy with an UPDATE statement.
SELECT Replace(births, '00', '')
or
update .....
Also you can use STUFF()
This function can be used for delete a certain length of the string and insert a new string in the deleted place.
Select STUFF ('199900', 5, 2, '')
--result 1900,1982,....
Ahsan Kabir Please remember to click Mark as Answer and Vote as Helpful on posts that help you. This can be beneficial to other community members reading the thread. http://www.aktechforum.blogspot.com/

Creation of DB Adaptert for calling stored procedure in MS SQL server

Hi,
I need to create a DB adapter to call a stored procedure in MS SQL Server.
I have gone thru the thread MS SQL Server database connection
It mentions that we need to use a command line utility for generating the wsdl and xsd for calling stored procedures in MS SQL server. Please provide information where to find this utility and how to use it.
Any links to tutorials are welcome.
Thanks !!.
Silas.

Command line is required for stored procedures, if you are using the basic options you don't need to worry.
(1) Download MS SQL Server 2005 JDBC Driver from Microsoft Site. http://msdn.microsoft.com/en-us/data/aa937724.aspx
(2) The download is self extracting exe file. Extract this into Program Files on your machine. It should create folder as "Microsoft SQL Server 2005 JDBC Driver"
(3) In above mentioned folder search for sqljdbc.jar copy this file into JDeveloper\JDBC\lib folder.
(4) Open JDeveloper/jdev/bin/jdev.conf file add following entry.
AddJavaLibPath C:/Program files/Microsoft SQL Server 2000 Driver for JDBC/lib
While executing this step make sure that your JDeveloper is closed.
(5) On command prompt go to J Developer folder and execute following command
jdev -verbose
This will open JDeveloper.
(6) Now go to JDeveloper > Connections > Database Connections > New Database Connection
(7) Select Third Party JDBC
(8) Specify MS Sql Server User Name, password and Role.
(9) In connection page specify following
- Driver Class: com.microsoft.sqlserver.jdbc.SQLServerDriver
- For class path browse to C:/Program files/Microsoft SQL Server 2000 Driver for JDBC/lib folder, select sqljdbc.jar add it as library.
- Specify URL as following.
jdbc:sqlserver://SERVERNAME:1433;databaseName=MSSQLDBNAME;
(10) Go to Test page and test it.
cheers
James

How to invoke a stored procedure on MS Sql Server with Java?

I started writing Enterprise Java Beans and created an ODBC dsn with MS Sql Server 2000 which I can access using jdbc:odbc:mySqlDSN. This all works fine using Java Sql Statements. What kind of Java/Java Sql statement can I use to invoke a stored procedure on the Sql Server? Is it possible to use ADO/ADO command objects with Java? Is it possible to import/implement Mdac2.6 ActiveX data objects libary in Java?
Thanks

Thanks all for your replies. I will search the api for callable statements. I am curious though, the reply that suggests using a prepared statement - can I put the name of a stored procedure in a prepared statment or is this just suggestions an action query like Insert Into, Update, Delete? Like with ADO you can say
cmdObject.CommandType = adStoredProcedure
cmdObject.CommandText = "NameOfStoredProc"
cmdObject.ExecuteNonQuery()
Once I am calling/importing/implementing the proper libraries/interfaces in Java, can a prepared statement reference a stored procedure as above?
Thanks

Ident_Current('TableName') not giving last inserted identity Value in SQL Server 2012

Hi ,
Ident_Current('TableName') is not giving last inserted identity Value in SQL Server 2012.
Latest Value of ID and Ident_Current is differing for that particular table.
Why its happening? Ident_Current value is greater than last inserted ID.
Please guide me why its happening?

Here is one example:
CREATE DATABASE PMCtest ON (NAME = 'PMCtest', FILENAME = 'B:\ProdDB-Unzipped\PMCtest.mdf', SIZE = 25GB )
LOG ON (NAME = 'PMCtest_log', FILENAME = 'B:\ProdDB-Unzipped\PMCtest.ldf', SIZE = 2GB)
CREATE TABLE identity_crisis (a int IDENTITY NOT NULL PRIMARY KEY,
b sysname NOT NULL CHECK (b <> 'sysschobjs'))
go
INSERT identity_crisis (b) VALUES ('First name')
INSERT identity_crisis SELECT name FROM sys.objects
go
SELECT MAX(a), ident_current('identity_crisis')
FROM identity_crisis
go
DROP TABLE identity_crisis
Because the second INSERT statement fails, the identity values from 2 to 12 are lost. This is a very intentional design to promote high concurrency. If you don't want gaps, you should not use IDENTITY, but roll your own.
Also, beware that ident_current returns database-wide value, so if you insert one row, and another process inserts another row before you interrogate ident_current, you will get the id for the row of the other process.
Erland Sommarskog, SQL Server MVP, [email protected]

Bit value in sql-server

hello frnds,
i want to insert bit value into sql-server 2000... using hibernate
what is the datatype for java bean.... ?
and how it is mappend in mapping xml....?
if any one has idea then please help me...
thanx in advance..
regards
PanDev84

The WPF MediaElement (unlike Silverlight's) doesn't support setting the Source to a Stream directly. You could try any of the workarounds suggested here:
http://stackoverflow.com/questions/7117589/using-mediaelement-to-play-video-from-stream
Othwerwise you could select the data (bytes) from the database into a byte[]:
http://stackoverflow.com/questions/13952261/insert-and-retrieve-bytearray-from-sql-server-in-c-sharp
...and then save the data to a temporary physical file and play this one as usual:
byte[] data = ...; //select from db
System.IO.File.WriteAllBytes("temp.wav", data);
var mediaPlayer = new MediaPlayer();
mediaPlayer.Open(new Uri("temp.wav", UriKind.Relative));
mediaPlayer.Play();
The MediaPlayer.Open method will read the file via a stream automatically.
Hope that helps.
Please remember to close your threads by marking helpful posts as answer and please start a new thread if you have a new question.

Reset Identity Column Value in SQL Server depending on whats being inserted

I have a SQL Server 2008 database with a table that has an ID field that has an identity increment and seed of 1.
This work great as each new record created automatically gets a new and distinct number.
Now when I insert certain new records I need to be able to make that value start at 10001 and then the next would be 10002 etc.
But this is only for certain inserts.
Is there a way to control the identity seed & increment on the fly via a query maybe?
So some inserts would follow 1,2,3,4,5,6,7 depending on whats being inserted
and others would follow 10001,10002,10003 depending on whats being inserted.
is this possible?

Make it a different field. Don't mess with your PK.
Make sure you have a plan for when you get 10000 records in this table.

Storing Unicode data in SQL Server 2000

hello,
I'm currently developing a website which must store Cyrillic characters in a SQL Server 2000 database. I know the database can store the data correctly because when I use the MS Front end to connect to the database I am able to copy/paste the text into the database columns correctly.
Retrieving the data from the database also works correctly and is displayed with the correct characters.
The problems I am having is that the text seems to get garbled during the SQL insert by the database driver. If I print out the SQL just before it is inserted the characters are still correct but once they get into the database they are wrong. I'm using the JDBC drive to connect to the database.
Any help would be appreciated,
Alan

I change the way I was storing the data from using unicode to getting the char values of each letter (ie \u1041 for Б) and storing them in the database. Then I parse the numbers when I bring them back out and combine them into the original String

How can I Use a Stored Procedure from Microsoft SQL Server?

Hi All,
Would like to use stored procedure as my data service in Visual Composer.
Our version is VC7.0 SP20.
Is stored procedure feasible? What is the system that i need to create in Enterprise Portal? Currently i have BI JDBC System which only allows me to search for tables from SQL Server.
Much appreciate your help.
Thanks & Regards,
Sarah

Hi Skif,
Referring to my post: JDBC System Connection VS BI JDBC System Connection
I do not even able to view list of stored procedure. I need help too..maybe you can show me how do you set the connection? What are the mandatory connection properties? I have filled up:
- Connection URL
- Driver class name
- User Administration: User, Admin (I did user mapping too)
The error message that i got is:
com.sap.guimachine.portalconnector.commandhandler.CommandException: Failed to connect to backend system. Check your system definition and user privileges.#
I had assigned full control to the DB user, and map my portal user to that DB user.
Many Thanks,
Sarah

DB Adapter calling stored procedure in MS SQL Server.

Hi
i am calling a stored procedure in a MS SQL Server.
I configured a Datasource for MS SQL Server driver as Oracle's MS SQL Server Driver(Type 4) Versions: 7.0 and later and provide all db details of sql server. I tested the data source and was getting success message. My BPEL code is just talking with input parameter and then calling the stored procedure in sql server. I was able to compile the code in jdeveloper and deploy to weblogic. But when i am testing the code. i am getting an error.
<env:Fault>
<faultcode>env:Server</faultcode>
<faultstring>Exception occured when binding was invoked.
Exception occured during invocation of JCA binding: "JCA Binding execute of Reference operation 'IDMService' failed due to: JCA Binding Component connection issue.
JCA Binding Component is unable to create an outbound JCA (CCI) connection.
Test:IDMService [ IDMService_ptt::IDMService(InputParameters,OutputParameters) ] : The JCA Binding Component was unable to establish an outbound JCA CCI connection due to the following issue: javax.resource.spi.IllegalStateException: [Connector:199176]Unable to execute allocateConnection(...) on ConnectionManager. A stale Connection Factory or Connection Handle may be used. The connection pool associated with it has already been destroyed. Try to re-lookup Connection Factory eis/sql/idm from JNDI and get a new Connection Handle.
Please make sure that the JCA connection factory and any dependent connection factories have been configured with a sufficient limit for max connections. Please also make sure that the physical connection to the backend EIS is available and the backend itself is accepting connections.
The invoked JCA adapter raised a resource exception.
Please examine the above error message carefully to determine a resolution.</faultstring>
<faultactor/>
<detail>
<exception>[Connector:199176]Unable to execute allocateConnection(...) on ConnectionManager. A stale Connection Factory or Connection Handle may be used. The connection pool associated with it has already been destroyed. Try to re-lookup Connection Factory eis/sql/idm from JNDI and get a new Connection Handle.</exception>
</detail>
</env:Fault>
What is the problem with connection . How cna i rectify it.
Thanks,
Venu Raja

followed the steps mentioned in meatalnk..
JCA-12563 - Unable to Establish an Outbound JCA CCI Connection [ID 1235943.1]
Metalink ID - 1235943.1

How to update the identity_column value in sql server 2008

Hi Team,
I have identity column based table. In the table i use insert the name as per the requirement and for ID i use to set the identity now i need to update the particular row id.
when i use to trying i'm getting the below error.
Msg 8102, Level 16, State 1, Line 4
Cannot update identity column 'CategoryID'.
Kindly help me out regarding this.
Thanks & Regards,
Ramesh R

You cannot update an IDENTITY column. You need to move the data into another table make the changes over there and insert the back the data into original table.
Best Regards,Uri Dimant SQL Server MVP,
http://sqlblog.com/blogs/uri_dimant/
MS SQL optimization: MS SQL Development and Optimization
MS SQL Consulting:
Large scale of database and data cleansing
Remote DBA Services:
Improves MS SQL Database Performance
SQL Server Integration Services:
Business Intelligence

Calling Managed CLR Stored Procedure from C++ SQL Server 2008 fails with Msg 2809

I am trying to call a stored procedure from C++ (VS 2010).
Here is what the stored procedure looks like:
public class Validate
[Microsoft.SqlServer.Server.SqlProcedure(Name = "ClientTest")]
public static void ClientTest(out String res )
{ res = "50";}
To create a stored procedure I deploy at first the assembly
USE [test]
GO
CREATE ASSEMBLY ClientTestAssembly
AUTHORIZATION testLogin
FROM 'C:\Users\test\Documents\Visual Studio 2010\Projects\TestCreationAssemblyCSharp\TestCreationAssemblyCSharp\bin\x64\Debug\TestCreationAssemblyCSharp.dll'
and call
USE test
GO
CREATE PROCEDURE ClientTest (@res NVARCHAR(40) OUTPUT)
AS
EXTERNAL NAME ClientTestAssembly.Validate.ClientTest
I can call my procedure direct in SQL Server Management Studio without any errors
declare @res nvarchar(10)
execute ClientTest @res OUTPUT
print @res
But when I try to call this procedure from C++ using CALL syntax
SQLBindParameter(sqlstatementhandle, 1, SQL_PARAM_OUTPUT, SQL_C_CHAR,
SQL_VARCHAR , 40, 0, version, sizeof(version) ,
&pcbValue);
res = SQLExecDirect(sqlstatementhandle, (SQLCHAR*)("{CALL ClientTest(?) }"), SQL_NTS);
I get the Error 2809 with SQLSTATE 42000 and with statement
The request for 'ClientTest'
(procedure) failed because 'ClientTest' is a procedure object.
However, if I wrap my CLR stored procedure into a T-SQL stored procedure, like
CREATE PROCEDURE myProc (@res NVARCHAR(40) OUTPUT)
AS
EXEC ClientTest @res OUTPUT
and call then myProc instead of ClientTest
res = SQLExecDirect(sqlstatementhandle, (SQLCHAR*)("{CALL myProc(?) }"), SQL_NTS);
everithing works fine and I get 50 as result.
I have no ideas what is the reason for that behavior.
Thanks very much for any suggestion.
Christina

I'm not an ODBC expert, but you might try following the sample here:
Process Return Codes and Output Parameters (ODBC)
To see if it also behaves differently for your CLR proc.
David
David http://blogs.msdn.com/b/dbrowne/

Storing 6 millions values in SQL server columns

Similar Messages

Maybe you are looking for