Uploading data file with specific encoding and other metadata

Hello everyone.
I've been recently testing Microsoft Codename "Data Hub". When select the option to upload a CSV or XLSX file to populate my database, it basically has two problems, in my opinion:
1. It would be great if I could select the file's encoding. If my file is in Spanish, it won't properly load latin characters (e.g. accents, or special characters). The information ends up showing with a bunch of weird characters.
2. It also doesn't let me select which character to use as data separator. It uses the comma as the default. Some of the data might contain commas, so I usually change the separator to a pipe in my CSV files.
Does anybody know any workarounds?
Thanks.

Hey there, and apologies. I wanted to reach out and see if this is still a problem for you.
I also wanted to check if this is a question about Azure SQL Database or something else?
Thanks Guy

Similar Messages

SQL Loader - CSV Data file with carraige returns and line fields

Hi,
I have a CSV data file with occasional carraige returns and line feeds in between, which throws my SQL loader script off. Sql loader, takes the characters following the carraige return as a new record and gives me error. Is there a way I could handle carraige returns and linefeeds in SQL Loader.
Please help. Thank you for your time.
This is my Sql Loader script.
load data
infile 'D:\Documents and Settings\user1\My Documents\infile.csv' "str '\r\n'"
append
into table MYSCHEMA.TABLE1
fields terminated by ','
OPTIONALLY ENCLOSED BY '"'
trailing nullcols
( NAME CHAR(4000),
field2 FILLER,
field3 FILLER,
TEST DEPT CHAR(4000)
)

You can "regexp_replace" the columns for special characters

Indesign server web service - where is the support to upload a file with the script and download the result?

Hi,
I am working on a POC that is supposed to convert indd files to pdf (i.e. using of course the indesign server). Basically I want to call the IDS Web Service (located on a different machine perhaps), pass in the input file, the conversion script and retrieve the result as part of the WebService call?
Browsing the documentation, examples, etc in the SDK, I couldn't see how the above can be achieved without the client handling the file transfer. Surely I must be missing something ...
thanks
Chris

What do you mean with POC?
InDesign Server is too precious (i.e. license cost) to waste its time with file transfers.
For a smaller scale, let a separate process (some http or smb server) on the same server hardware handle the files - so that the InDesign Server can access them on the fastest local volume.
If you plan for bigger, use a dedicated server for file sharing (your choice of SMB, NFS or whatever), where the input files are prepared by the client process, so that your load balancer can immediately point the next free instance of the InDesign server farm to the file. In that case be prepared for some try and error - high speed file sharing can be tricky with files written from one side not yet visible or incomplete to the other side, locking problems, Unicode file name trouble, unexpected time stamps and so forth.
Btw, there is also an InDesign Server forum which would be more appropriate for such discussions.
Dirk

Writing to file with specific encoding in unix

hi,
I want to write html files which contain Turkish characters in unix operating system.
I'm currently using FileWriter to write the files.
I'm getting the file content from the database and I can see that the characters seem to be fine but when I write them into an html file, they are displayed with the question mark character (?).
What can I do about this?
Thanks

FileWriter uses the default character encoding of whatever platform your program is running on. Beside the fact that the default encoding may vary from one machine to another, it may not be suitable for storing non-ASCII (Turkish) characters.
Rather than using FileWriter, you can use an OutputStreamWriter and a FileOutputStream, which allows you to specify the character encoding you want. UTF-8 is generally a good choice, since it can encode just about anything. For example:FileOutputStream fos = new FileOutputStream("/some/file/name");
OutputStreamWriter osw = new OutputStreamWriter(fos, "utf-8");
osw.write("your text with Turkish characters...");Geoff

Yosemite finder crashes with previewig .txt and other files

Hello,
under Yosemite the Finder crashes with previewig .txt and other files.
Is this just my problem? Help please.
Thank you very much.
Regards.

Launch the Console application in any of the following ways:
☞ Enter the first few letters of its name into a Spotlight search. Select it in the results (it should be at the top.)
☞ In the Finder, select Go ▹ Utilities from the menu bar, or press the key combination shift-command-U. The application is in the folder that opens.
☞ Open LaunchPad and start typing the name.
Step 1
For this step, the title of the Console window should be All Messages. If it isn't, select
          SYSTEM LOG QUERIES ▹ All Messages
from the log list on the left. If you don't see that list, select
          View ▹ Show Log List
from the menu bar at the top of the screen.
In the top right corner of the Console window, there's a search box labeled Filter. Initially the words "String Matching" are shown in that box. Enter the name of the crashed application or process. For example, if Safari crashed, you would enter "Safari" (without the quotes.)
Each message in the log begins with the date and time when it was entered. Select the messages from the time of the last crash, if any. Copy them to the Clipboard by pressing the key combination command-C. Paste into a reply to this message by pressing command-V.
☞ The log contains a vast amount of information, almost all of which is irrelevant to solving any particular problem. When posting a log extract, be selective. A few dozen lines are almost always more than enough.
Please don't indiscriminately dump thousands of lines from the log into this discussion.
Please don't post screenshots of log messages—post the text.
☞ Some private information, such as your name, may appear in the log. Anonymize before posting.
Step 2
In the Console window, select
          DIAGNOSTIC AND USAGE INFORMATION ▹ User Diagnostic Reports
(not Diagnostic and Usage Messages) from the log list on the left. There is a disclosure triangle to the left of the list item. If the triangle is pointing to the right, click it so that it points down. You'll see a list of crash reports. The name of each report starts with the name of the process, and ends with ".crash". Select the most recent report related to the process in question. The contents of the report will appear on the right. Use copy and paste to post the entire contents—the text, not a screenshot.
I know the report is long, maybe several hundred lines. Please post all of it anyway.
If you don't see any reports listed, but you know there was a crash, you may have chosen Diagnostic and Usage Messages from the log list. Choose DIAGNOSTIC AND USAGE INFORMATION instead.
In the interest of privacy, I suggest that, before posting, you edit out the “Anonymous UUID,” a long string of letters, numbers, and dashes in the header of the report, if it’s present (it may not be.)
Please don’t post other kinds of diagnostic report—they're very long and rarely helpful.

I installed Lion and now seem to have problems with Microsoft Silverlight and other plug ins and applications. I haven't ever used time machine to back up (my bad I know). Is there a way to go back to snow leopard with messing up all my files and my set

I installed Lion and now seem to have problems with Microsoft Silverlight and other plug ins and applications. I haven't ever used time machine to back up (my bad I know). Is there a way to go back to snow leopard with messing up all my files and my set?

Are you using the latest version of Silverkeeper? - v.2.0.2 is stated to be compatible with Snow Leopard.
http://www.lacie.com/silverkeeper/
If it's messing things up you could try asking LaCie Support for assistance.

Upload multiple files WITH correct pairs of form fields into Database

In my form page, I would like to allow 3 files upload and 3 corresponding text fields, so that the filename and text description can be saved in database table in correct pair. Like this:
INSERT INTO table1 (filename,desc) VALUES('photo1.jpg','happy day');
INSERT INTO table1 (filename,desc) VALUES('photo2.jpg','fire camp');
INSERT INTO table1 (filename,desc) VALUES('photo3.jpg','christmas night');
However, using the commons fileupload, http://commons.apache.org/fileupload/, I don't know how to reconstruct my codes so that I can acheieve this result.
if(item.isFormField()){
}else{
}I seems to be restricted from this structure.
The jsp form page
<input type="text" name="description1" value="" />
<input type="file" name="sourcefile" value="" />
<input type="text" name="description2" value="" />
<input type="file" name="sourcefile" value="" />The Servlet file
package Upload;
import sql.*;
import user.*;
import javax.servlet.*;
import javax.servlet.http.*;
import java.util.Map;
import java.util.HashMap;
import java.util.Date;
import java.util.List;
import java.util.Iterator;
import java.io.File;
import java.io.PrintWriter;
import java.io.IOException;
import java.text.SimpleDateFormat;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.*;
public class UploadFile extends HttpServlet {
private String fs;
private String category = null;
private String realpath = null;
public String imagepath = null;
public PrintWriter out;
private Map<String, String> formfield = new HashMap<String, String>();
//Initialize global variables
public void init(ServletConfig config, ServletContext context) throws ServletException {
    super.init(config);
//Process the HTTP Post request
public void doPost(HttpServletRequest request,HttpServletResponse response) throws ServletException, IOException {
    request.setCharacterEncoding("utf-8");
    response.setCharacterEncoding("utf-8");
    response.setContentType("text/html");
    PrintWriter out = response.getWriter();
    Thumbnail thumb = new Thumbnail();
    fs = System.getProperty("file.separator");
    this.SetImagePath();
     boolean isMultipart = ServletFileUpload.isMultipartContent(request);
     if(!isMultipart){
      out.print("not multiple part.");
     }else{
         FileItemFactory factory = new DiskFileItemFactory();
         ServletFileUpload upload = new ServletFileUpload(factory);
         List items = null;
         try{
            items = upload.parseRequest(request);
         } catch (FileUploadException e) {
            e.printStackTrace();
         Iterator itr = items.iterator();
         while (itr.hasNext()) {
           FileItem item = (FileItem) itr.next();
           if(item.isFormField()){
              String formvalue = new String(item.getString().getBytes("ISO-8859-1"), "utf-8");
              formfield.put(item.getFieldName(),formvalue);
              out.println("Normal Form Field, ParaName:" + item.getFieldName() + ", ParaValue: " + formvalue + "<br/>");
           }else{
             String itemName = item.getName();
             String filename = GetTodayDate() + "-" + itemName;
             try{
               new File(this.imagepath + formfield.get("category")).mkdirs();
               new File(this.imagepath + formfield.get("category")+fs+"thumbnails").mkdirs();
               //Save the file to the destination path
               File savedFile = new File(this.imagepath + formfield.get("category") + fs + filename);
               item.write(savedFile);
               thumb.Process(this.imagepath + formfield.get("category") +fs+ filename,this.imagepath + formfield.get("category") +fs+ "thumbnails" +fs+ filename, 25, 100);
               DBConnection db = new DBConnection();
               String sql = "SELECT id from category where name = '"+formfield.get("category")+"'";
               db.SelectQuery(sql);
                while(db.rs.next()){
                  int cat_id = db.rs.getInt("id");
                  sql = "INSERT INTO file (cat_id,filename,description) VALUES ("+cat_id+",'"+filename+"','"+formfield.get("description")+"')";
                  out.println(sql);
                  db.RunQuery(sql);
             } catch (Exception e){
                e.printStackTrace();
        HttpSession session = request.getSession();
        UserData k = (UserData)session.getAttribute("userdata");
        k.setMessage("File Upload successfully");
        response.sendRedirect("./Upload.jsp");
//Get today date, it is a test, actually the current date can be retrieved from SQL
public String GetTodayDate(){
    SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
    String today = format.format(new Date());
    return today;
//Set the current RealPath which the file calls for this file
public void SetRealPath(){
    this.realpath = getServletConfig().getServletContext().getRealPath("/");
public void SetImagePath(){
    this.SetRealPath();
    this.imagepath = this.realpath + "images" +fs;
}Can anyone give me some code suggestion? Thx.

When one hits the submit button - I then get a 404 page error.What is the apaches(?) error log saying? Mostly you get very useful information when looking into the error log!
In any case you may look at how you are Uploading Multiple Files with mod_plsql.

Uploading Data File to Company Database

I've created a form using iWork Numbers on my iPad.
If I create a data file from specific data in my form is there a way to upload the data file from my iPad directly to a company database over the internet.
I don't want to have use iTunes or MobileMe because it seems that the data would then have to be manually forwarded to the database. At least that's my perception but I could be wrong.
I would appreciate any help on directly forwarding data from an iPad to a company database via the internet.
Thanks

The only thing I can think of off the top of my head is email as an excel file. You would want to email it to yourself and open in excel, then use that (unmodified) file as a template for your IT to set up an email and script that would take the emailed file and import into the proper table(s).
Jason

How to upload data from excel to SAP and options to be used

How to upload data from excel to SAP and options to be used
thank you,
Regards,
Jagrut Bharatkumar shukla

Hi Jagrut,
You can use gui_upload.
chk the sample program mentioned below.
REPORT ZFTP .
DATA: BEGIN OF I_FILE OCCURS 0,
DATA(2000) TYPE C,
END OF I_FILE.
DATA: BEGIN OF I_FILE2 OCCURS 0,
DATA(2000) TYPE C,
END OF I_FILE2.
DATA: W_COUNT TYPE I.
PARAMETERS: P_FILEN TYPE STRING,
P_FILE2 TYPE STRING,
P_NUM(4) TYPE N..
AT SELECTION-SCREEN ON VALUE-REQUEST FOR P_FILEN.
PERFORM F_FILE_GET USING P_FILEN TEXT-G01.
AT SELECTION-SCREEN ON VALUE-REQUEST FOR P_FILE2.
PERFORM F_FILE_GET USING P_FILE2 TEXT-G01.
START-OF-SELECTION.
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
filename = P_FILEN
FILETYPE = 'ASC'
HAS_FIELD_SEPARATOR = 'X'
HEADER_LENGTH = 0
READ_BY_LINE = 'X'
DAT_MODE = ' '
CODEPAGE = ' '
IGNORE_CERR = ABAP_TRUE
REPLACEMENT = '#'
CHECK_BOM = ' '
VIRUS_SCAN_PROFILE =
NO_AUTH_CHECK = ' '
IMPORTING
FILELENGTH =
HEADER =
tables
data_tab = I_FILE
IF SY-SUBRC <> 0.
MESSAGE E024(Z1).
ENDIF.
LOOP AT I_FILE.
W_COUNT = W_COUNT + 1.
IF NOT W_COUNT > P_NUM.
MOVE I_FILE TO I_FILE2.
APPEND I_FILE2.
ENDIF.
ENDLOOP.
CALL FUNCTION 'GUI_DOWNLOAD'
EXPORTING
BIN_FILESIZE =
filename = P_FILE2
FILETYPE = 'ASC'
APPEND = ' '
WRITE_FIELD_SEPARATOR = 'X'
HEADER = '00'
TRUNC_TRAILING_BLANKS = ' '
WRITE_LF = 'X'
COL_SELECT = ' '
COL_SELECT_MASK = ' '
DAT_MODE = ' '
CONFIRM_OVERWRITE = ' '
NO_AUTH_CHECK = ' '
CODEPAGE = ' '
IGNORE_CERR = ABAP_TRUE
REPLACEMENT = '#'
WRITE_BOM = ' '
TRUNC_TRAILING_BLANKS_EOL = 'X'
WK1_N_FORMAT = ' '
WK1_N_SIZE = ' '
WK1_T_FORMAT = ' '
WK1_T_SIZE = ' '
IMPORTING
FILELENGTH =
tables
data_tab = I_FILE2
FIELDNAMES =
*& Form F_FILE_GET
text
-->P_P_FILEN text
-->P_TEXT_G01 text
FORM F_FILE_GET USING L_FILENA L_TEXT.
CALL FUNCTION 'WS_FILENAME_GET'
EXPORTING
DEF_FILENAME = ' '
DEF_PATH = ' '
MASK = ',.,*.TXT.'
MODE = 'O'
TITLE = L_TEXT
IMPORTING
FILENAME = L_FILENA
rc =
EXCEPTIONS
INV_WINSYS = 1
NO_BATCH = 2
SELECTION_CANCEL = 3
SELECTION_ERROR = 4
OTHERS = 5
IF SY-SUBRC <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
Reward if helpful.
Regards,
Harini.S

How to load unicode data files with fixed records lengths?

Hi!
To load unicode data files with fixed records lengths (in terms of charachters and not of bytes!) using SQL*Loader manually, I found two ways:
Alternative 1: one record per row
SQL*Loader control file example (without POSITION, since POSITION always refers to bytes!)<br>
LOAD DATA
CHARACTERSET UTF8
LENGTH SEMANTICS CHAR
INFILE unicode.dat
INTO TABLE STG_UNICODE
TRUNCATE
A CHAR(2) ,
B CHAR(6) ,
C CHAR(2) ,
D CHAR(1) ,
E CHAR(4)
) Datafile:
001111112234444
01NormalDExZWEI
02ÄÜÖßêÊûÛxöööö
03ÄÜÖßêÊûÛxöööö
04üüüüüüÖÄxµôÔµ Alternative2: variable length records
LOAD DATA
CHARACTERSET UTF8
LENGTH SEMANTICS CHAR
INFILE unicode_var.dat "VAR 4"
INTO TABLE STG_UNICODE
TRUNCATE
A CHAR(2) ,
B CHAR(6) ,
C CHAR(2) ,
D CHAR(1) ,
E CHAR(4)
) Datafile:
001501NormalDExZWEI002702ÄÜÖßêÊûÛxöööö002604üuüüüüÖÄxµôÔµ Problems
Implementing these two alternatives in OWB, I encounter the following problems:
* How to specify LENGTH SEMANTICS CHAR?
* How to suppress the POSITION definition?
* How to define a flat file with variable length and how to specify the number of bytes containing the length definition?
Or is there another way that can be implemented using OWB?
Any help is appreciated!
Thanks,
Carsten.

Hi Carsten
If you need to support the LENGTH SEMANTICS CHAR clause in an external table then one option is to use the unbound external table and capture the access parameters manually. To create an unbound external table you can skip the selection of a base file in the external table wizard. Then when the external table is edited you will get an Access Parameters tab where you can define the parameters. In 11gR2 the File to Oracle external table can also add this clause via an option.
Cheers
David

How to create PDF from text file with specific layout?

I wanted to create the pdf from text file in specific layout - Landscape orientation and JIS B3 Page size while at Adobe Acrobat Pro.
In past, I could do a right click on a text file (desktop area) and select print to print out the document into .pdf BUT only if I set the Adobe PDF to Landscape and JIS B3 Page size BEFORE. And I could only do 15 text documents at once.
I wanted to see if I could do the create the pdf from text file with specific layout in Adobe Acrobat without having to go to Control Panel to preset the Adobe PDF to specific layout at every time. I would have to set Adobe PDF back to normal layout after I'm done with these pdf print outs. I do lots of pdfs in normal layout. Sometimes I would forget to do that.
So, How do I do that?

No such luck. It would output the contents in letter size even in JIS B3 Page layout at MS word.
Is there a script or action where I could set the orientation and page size before creating PDF on these text files?

Want attach the XML data file with layout template in Oracle 10g

Hi All,
I need a help from you genius guys.
I am genrating reports in BI with xml the procedure which I am following is as below.
1. generating XML from the RDF
2. creating a template in .rtf format
3.after that loading the xml to the template then getting the required report.
This all is doing through the given buttons
But now my requirement is to create the gui from user can select the report and get the desire output file so how we would be able to attach the XML data file with layout template in Oracle 10g.
If you require more detail please let me knnow.
Thanks,
Harry

I am not using Oracle apps.
I am using oracle 10g reports and I get something in it like one patch I downloded and one java code is having which creates the batch file ...still I am working on it ..
If you will get some please share so that it will be helpful.
Thanks,
Harry

Writing a report file with specific information

Hi, all:
I need to write a text file with specific information in it. I'm going to copy it into Excel, so the easier the format, the better (perhaps CSV format?). I've never done this before, so here's the method I need to capture the information from:
     public void step() {
          greenColorStorage = new ArrayList();
          magentaColorStorage = new ArrayList();
          System.out.println( "==> Model step " + getTickCount() );// Checking step #
          for (int i = 0; i < soldierList.size (); i++) {
               Soldier s = (Soldier) soldierList.get (i);
               s.step();
          Collections.shuffle(soldierList);
          System.out.println("Shuffling collection now.");
          dsurf.updateDisplay ();
          System.out.println("Model.step() finished.");
          for (int i = 0; i < soldierList.size (); i++) {
               Soldier s = (Soldier) soldierList.get (i);
               if (s.getMyColor() == Color.green ) {
                    greenColorStorage.add(s);
               if (s.getMyColor() == Color.magenta ) {
                    magentaColorStorage.add(s);
          if (magentaColorStorage.size() == 0 || greenColorStorage.size() == 0 ) {
               stop();
     }In this method, I need to capture this information: There are soldiers on the Green team and on the Magenta team. I need to know how many there are originally, how many in this time step, at what time step the model stops, and how many of the winning color team are left when the model stops. How do I do this? If you can point me to a specific thread that shows how to write a report file like this, I'd be grateful, since I couldn't find one when I searched. Otherwise, I'd be grateful for examples. I'm in the middle of doing some research, and I have the feeling that I'll need to write report files in the future, so if you'd explain any code you provide, I would be quite grateful--I need to understand what I'm doing as well as simply using your code so that I can really learn what I'm doing when I do this again. Thanks very much!

FileOutputStream
public FileOutputStream(File file,
                        boolean append)
                 throws FileNotFoundException
Creates a file output stream to write to the file represented by the specified File object.
If the second argument is true, then bytes will be written to the end of the file
rather than the beginning.
*/

LabVIEW created .DAT file with saving only new data on existing files

I can successfully create and save a .DAT file using LabView and Diadem. The save command I am using (in Labview) only overwrites existing files, I would like it to save only the new data from the last save. Overall I would like to save the data every 1hr and since my test will last for many hours, the further along in the test the longer it would take to save the full file. I know Diadem has the capability to save in such a manner, so I have thought about after I write to Diadem have a small Diadem application save periodically, but I haven't found a way to save the .DAT file, with out any signals coming into the save function. I have attached the example .DAT file creater and saver I am using to figure it out. thanks for your help!

Howdy New Guy -
I am not sure if you are looking to append to a file, or if you would
like to overwrite a file. In either case, you can take a look at
the "Append To File.vi" example for DIAdem.
Either way, you may be experiencing problems because you are only using
the "DIAdem Simple File Write.vi" (found on the DIAdem >> DAT
Files palette). Instead, you can use the "DIAdem File Write.vi"
(located on teh DIAdem >> DAT Files >> DAT Files Advanced
palette). This VI can be set to overwrite or append to file.
I hope this helps in your development!
Andrew W. || Applications Engineer

I need to upload my files from one directory into other directory.

i need to upload my files from one directory into other directory.( all its contents,folders,sub folders, files etc)
for this task
suppose my source folder is C:/test/test1/test2/test3
and my destination is D:/
first i have to check whether my dest dirctory already contains source directory..and its files
if it contains the same structure then i have to replace the destination dir structure to source dir structure( to upadte changes)
if it(dest dir) not contains source dir ..means it doesn't contain source folder then
i have to create same source folder directory structure ( test.test1/test2/test3...) at destination(D:/) and copy all its contents frm source directory...
( means i don;t want to create manually same dir structure as source at destnation dir to copy all contents)

I believe you need to read up on the class File.java (it comes with java).
Some examples:
File file = new File("C:/myDirectory");
if(file.isDirectory())
System.out.println("this is a directory");
if(file.isFile())
System.out.println("this is a file").
You will then need to read up on other classes to read/write to the file such as FileReader and FileWriter.

Uploading data file with specific encoding and other metadata

Similar Messages

Maybe you are looking for