Help: Need to finding Best way of Processing XML feeds into MySQL

Hi,
I wrote a bit of java code a couple of years ago to take an xml feed and load into MySQL.
It's ok for small files but memory intensive as the file gets larger.
1) Loads the XML into memory
2) Process nodes sequentially usiung via Cached XML Parser (xalan) ... memory
3) Writes each record ... slow
It needs updating and the xml files are bigger.
Any clues on how to do this efficiently?
Cheers,
Mark

markied wrote:
Load into in MySQL could take two steps:
1) bulk load into flat/generic xml structure in MySQL, if poss?
2) cache reading from flat/generic xml structure from MySQL and rewriting to new schema.I don't understand what either of those things mean so it's pretty hard to suggest alternatives to them.
If you're running out of memory then I assume you must be using DOM. I would have thought that sequential parsers like SAX and StAX would work for loading an XML document into an SQL database, no matter what database structures were involved.

Similar Messages

Need to find a way to convert mts files to aFinal Cut compatible format --

I hired someone to shoot something for me recently and he used AVCHD. However, he only copied the mts files and did not save the entire folder structure, which unfortunately is what Final Cut requires (at least that's my understanding of it). Basically I need to find a way to convert mts files into a Final Cut compatible file format.
Any ideas out there? Thanks in advance.

Forum search is your friend. The usual answer you'll find is ClipWrap, transcodes .mts to whatever flavor of ProRes you want.

I need to find a way to put emails on a personal folder that won't take up all my email space.

I'm on a mac and I need to find a way to put emails on a personal folder that won't take up all my email space. I've tried 3 times to set them up, but I keep getting the "almost full" message every night.

Can you tell us exactly what the "almost full" message says? Is it referring to your mailbox (ie, email server space), or your hard drive?
If your email server is almost full, you can store folders and messages on your Mac. In Mail, go to the Mailbox menu, click New Mailbox, and choose the location "On My Mac". Then drag folders into that new mailbox to move them to your local computer, and off the email server.
If your hard drive is full, this won't help, and you need a different approach.
What version of OS X are you running? It helps us for you to put that information into your profile, since there are big differences between versions of OS X.
Matt

Can anyone tell me how to get rid of tlb search? it unfortunately entered into my mac. Whenever I use safari, a pop-up window opens with some ads. Please help me to find a way to remove this malware.

Can anyone tell me how to get rid of tlb search? It unfortunately entered into my mac. Whenever I use safari, a pop-up window opens with some ads. Please help me to find a way to remove this malware.

You installed the "DownLite" trojan, perhaps under a different name. Remove it as follows.
Malware is constantly changing to get around the defenses against it. The instructions in this comment are valid as of now, as far as I know. They won't necessarily be valid in the future. Anyone finding this comment a few days or more after it was posted should look for more recent discussions or start a new one.
Back up all data.
Triple-click anywhere in the line below on this page to select it:
/Library/LaunchAgents/com.vsearch.agent.plist
Right-click or control-click the line and select
          Services ▹ Reveal in Finder (or just Reveal)
from the contextual menu.* A folder should open with an item named "VSearch" selected. Drag the selected item to the Trash. You may be prompted for your administrator login password.
Repeat with each of these lines:
/Library/LaunchDaemons/com.vsearch.daemon.plist
/Library/LaunchDaemons/com.vsearch.helper.plist
/Library/LaunchDaemons/Jack.plist
Restart the computer and empty the Trash. Then delete the following items in the same way:
/Library/Application Support/VSearch
/Library/PrivilegedHelperTools/Jack
/System/Library/Frameworks/VSearch.framework
~/Library/Internet Plug-Ins/ConduitNPAPIPlugin.plugin
Some of these items may be absent, in which case you'll get a message that the file can't be found. Skip that item and go on to the next one.
From the Safari menu bar, select
          Safari ▹ Preferences... ▹ Extensions
Uninstall any extensions you don't know you need, including any that have the word "Spigot" or "Conduit" in the description. If in doubt, uninstall all extensions. Do the equivalent for the Firefox and Chrome browsers, if you use either of those.
This trojan is distributed on illegal websites that traffic in pirated movies. If you, or anyone else who uses the computer, visit such sites and follow prompts to install software, you can expect much worse to happen in the future.
You may be wondering why you didn't get a warning from Gatekeeper about installing software from an unknown developer, as you should have. The reason is that the DownLite developer has a codesigning certificate issued by Apple, which causes Gatekeeper to give the installer a pass. Apple could revoke the certificate, but as of this writing, has not done so, even though it's aware of the problem. This failure of oversight is inexcusable and has compromised both Gatekeeper and the Developer ID program. You can't rely on Gatekeeper alone to protect you from harmful software.
*If you don't see the contextual menu item, copy the selected text to the Clipboard by pressing the key combination command-C. In the Finder, select
          Go ▹ Go to Folder...
from the menu bar and paste into the box that opens by pressing command-V. You won't see what you pasted because a line break is included. Press return.

Help needed in Finding Download location for Sun One Portal 7

Hi,
help needed for finding download location for Sun ONE Portal 7. I tried to find in Oracle Download page ,
http://www.oracle.com/us/sun/sun-products-map-075562.html, But unable to find.
Please share the link for download location.
I am totally new in Sun ONE Portal.
Thanks,
Edited by: 945439 on Oct 5, 2012 3:41 AM

try edelivery.oracle.com under sun products.

I have 2 screens at work and 1 screen at home. I was working on a power point presentation on 2 monitors. My window is now "stuck" when I Power Point. I cannot drag it back to a display or find any way to move it back into my view. Help!!!

I have 2 screens at work and 1 screen at home. I was working on a power point presentation on 2 monitors with the presentation on monitor externally hooked up to my computer. My window is now "stuck" when I open Power Point. I cannot drag it back to a display or find any way to move it back into my view. Help!!!

Here is what worked for me:
My usb hub, being usb2, was too fast. I moved the wire to a usb port directory on my pc. That is a usb1 port which is slow enough to run your snyc.

My iphone has been lost in my house and i need to find a way to track it and its offline. it wont connect to the internet so there isnt a way to use find my iphone. i just want to play a sound on it. is there any way i can get a app that connects my ipod

my iphone has been lost in my house and i need to find a way to track it and its offline. it wont connect to the internet so there isnt a way to use find my iphone. i just want to play a sound on it. is there any way i can get a app that connects my ipod from a laptop or a different way to find it.

If your phone is offline, the ONLY way to find it is to look for it the old fashioned way. Your phone CANNOT be located electronically if it's offline.

What is the best way to turn line art into a 3D image in Illustrator? Thanks for your help.

What is the best way to turn line art into a 3D image in Illustrator? Thanks for your help.

What version of Illustrator? What do you mean by "Line Art"? What do you mean by "3d image"? Post images showing your starting artwork and an example of what you are trying to achieve.
If you have Illustrator 12 or higher you can extrude or lathe a simple group or path using Effects > 3D > [Pick one]. You must start with vector data, not a placed raster image. You are very limited in what you can do, and you cannot model multiple objects within one 3D world. Each object you extrude/revolve/rotate is its own world, with its own perspective and camera. You can also make any art into a Symbol, then map that symbol onto any 3D surface you create. The results are, at best, acceptable, and often total garbage.

I need to find a way to convert .WMA files to .MP3 format on Lion

I need to find a way to convert .WMA files to .MP3 format; with Snow Leopard, I could do that using iTunes, but since upgrading to Lion, I can convert the files to a .mov format in Quick Time, but cannot move the file into iTunes in order to convert it to .MP3. If you can suggest a work-around or an app that will work with Lion, would be much appreciated.

Haven't used it myself, but worth a look http://www.macupdate.com/app/mac/27103/all2mp3
Stefan

Best way to extract XML data to DB

Hello,
In our work we need to extract data from XML documents into a database. Here are some extra notes:
+ There is no constant schema for the XML documents.
+ Some processing on the XML files is required.
+ The XML documents are very big.
+ The database is Oracle 8i or 9i (Enterprise Edition both) and our framework is NET.
Our questions are:
1. What is the best way to extract the data into the database under the above circumstances ?
2. In case there was a constant schema for the XML documents, would there be a better way ?
3. Is writing the data to text files first, and then loading it via SQL-Loader is an effective way ?
Any thoughts would be welcome. Thanks.

Hi Nicolas,
The answer depends on your actual storage method (binary, OR, CLOB?), and db version.
You can try XMLTable, it might be better in this case :
SELECT x.elem1, x.elem2, ... , x.elem20
FROM your_table t
   , XMLTable(
      '/an/xpath/to/extract'
      passing t.myxmltype
      columns elem1 varchar2(30) path 'elem1'
            , elem2 varchar2(30) path 'elem2'
            , elem20 varchar2(30) path 'elem20'
     ) x
;

What is the best way to insert massive data into an existing excel file?

dear gurus,
i am wondering that what is the best way to insert massive data into an existing excel file, more performance perspective.
the file is read from BDS , and we want to insert data into it .
the way i can think of is
1. OLE AUTOMATION
   i think performance will be a big problem
2. Office integration
    i am not sure it's facing the same performance issue ?
3 . XXL_SIMPLE_API/FULL_API
    I am not sure whether they can insert data into an existing excel file?
could you please give me some advices?
br.
jun

Hi,
If you want to APPEND data( add data to an existing excel file) from SAP, then use GUI_DOWNLOAD fm with APPEND = 'X' paramter.
Best regards,
Prashant

HT204053 I need to find my iPhone how do I sign into my cloud to find it?

I need to find my iPhone how do I sign into my cloud to find it?

Howdy Danny,
You should be able to find that lost iPhone with the Find My iPhone. It will let you locate the iPhone on a map or play a tone in case it’s under a couch cushion.
To learn more about how it works and the features, take a look at this article:
iCloud: Find My iPhone overview
http://support.apple.com/kb/ph2696
Thank you for using Apple Support Communities.
Cheers,
Sterling

Need help finding best way to import to iMovie

I've been playing around with this for a bit and decided to open the question up to you guys. I have old home movies on VHS. I don't have a miniDV camcorder or a digital converter. I do however have a Tivo DVR with all modern inputs/outputs (I believe...S Video..DV). Would the best way to move the VHS's to iMovie be to burn them to DVD using the tivo DVR and then converting the files into a compatible format for iMovie? Or is there an easier way to connect the tivo DVR to my macbook? I want to keep the best quality video as possible.
Thanks guys!

would a wire like this help me?
I doubt it. You want FireWire.
Take look at the Canopus ADVC300. It comes with a nice Macintosh application that works flawlessly with iMovie 06.
http://www.canopus.com/products/ADVC300/index.php
Yes, it does cost more but it works.

I need to know best way to edit MXF files, with least rendering, is that converting the MXF or Log and Transfer? the end product is dvd, but I have lots and lots of these projects to do.

I am about to comence my Edit, and was told of Mpeg Streamclip, however this doesn't seem to work with my Canon XF305 Clips, the reason I was told to use it was so I wouldn't have to Render in Final Cut Pro all the time.. I do have the Canon Disc with the Plug in Utility and the update from their site.
Is it best to Convert these MXF files first and import them into FCP or use 'Log and Transfer', so far I am unable to do either!!
The projects on the timeline will have introduction .Movs already on it as a Template to re-use again and again with the new Footage ( MXF Files ) to be added and edited with transitions etc..This process is repeated again and again but with different MXF clips each time.
I am looking for the quickest and best way for workflow as there are 80 or more projects to do, each with 10 or so clips to be added to the timeline and Exported for DVD...Will I have to then Compress to fit onto DVD's??
I know Streamclip is an Mepeg 2 app but I thought the Canon XF305 Shot as mpeg2....
Is Log and Transfer my only Option?
Basically I need to know the quickest for Render purposes, and of course, the best way so the files aren't so huge etc etc.
Please, any suggestions would be greatly received....

We only work with PR422 files in FC7. We use Shane's workflow with all 5D & MXF footage.
Make your life easier and stick to the ProRes family for your footage and edit sequences. Avoid H264 & convert any you have to ProRes.
Unless you are considering Blu-Ray the only way to deliver video on a DVD is to convert it to SD.
Depending on the length of your show you can make a Blu-Ray in Compressor but you have very limited authoring options when compared to DVD Studio Pro.
Compressor can also convert your HD to SD mpg-2 & Ac3 files. The best results usually come from setting the frame controls in the mpg-2 setting of Compressor to best. Depending on the the length of the show and your Qmaster settings this can take quite a while.
I personally find Compressor's HD to SD DVD conversion varies. Sometimes it is wonderful and sometimes not so much. I prefer the following as it is usually is faster and very consistent.
Export a quicktime movie (not QT conversion) of your final sequence
Import the exported HD file back into FC.
Create a SD 16x9 sequence and then drop in the recently exported HD seq and render. Rendering the sequence basically has FC doing the HD to SD conversion. Export the SD sequence and then convert to mpg-2 & ac3 in Compressor. While this does take time I still find it faster than Compressor with Qmaster utilizing all cores.
Another option is using jobs in Compressor
http://pixelcorps.cachefly.net/macbreak-154-540p-h264.mov
In this workflow you would have Compressor convert the HD file to SD and then through jobs convert the SD file to mpg2 & Ac3.
In theory the jobs version could be faster (based on a number of variables- length, Qmaster, etc) but I haven't had time to compare the two.

Best way to process out-of-gamut colors

Hi,
My usual colorspace whilst processing RAW images is Srgb, mainly because I print most shots & my printer requires Srgb for this.
The problem is that I experience "out-of-gamut" warnings within ACR and would appreciate some advice on the best way to deal with this - my options seem to be:
continue in Srgb & allow color to be clipped
swap to a wider colorspace (lab/Argb), process the image & tthen convert to Srgb for print
Ultimately though I need to output in Srgb so does it make any difference processing in the wider space as I guess the out-of-gamut color is going to be clipped at some stage anyway ?
thanks for any advice
Simon

simonkit wrote:
My usual colorspace whilst processing RAW images is Srgb, mainly because I print most shots & my printer requires Srgb for this.
The problem is that I experience "out-of-gamut" warnings within ACR and would appreciate some advice on the best way to deal with this - my options seem to be:
What printer are you using? Printers are nonlinear devices and as far as I know, no printer has sRGB as its native color space. Some printing services such as those offered by Walgreen and Walmart expect the image to be in sRGB and then convert behind the scenes to the native printer space. Walgreen often uses the Fuji Frontier and expects sRGB input. Some labs that cater to serious users will give you profiles for the Frontier. Some Costco stores use the Frontier and profiles are available on DryCreek.com. As Mylenium has pointed out in general for printers, the gamut of the Frontier is smaller than sRGB. However, their shapes are different and many colors in the sRGB gamut are not in the Frontier gamut. Conversely, a few colors in the printer gamut are outside of the Frontier gamut.
Here is a 3D gamut plot of sRGB in solid color and a Fuji Frontier printer from a lab in my area shown as a wireframe. The gamut of sRGB is wider in the blues and magentas and these colors that fit in sRGB would be clipped on output to this printer and you would have no way of knowing that in the absence of a profile for the printer. A few yellows in the gamut of the printer are outside the gamut of sRGB, and these could have been saved if you had used a wider space such as aRGB or ProPhotoRGB.
My suggestion is that you find a printer for which a profile is available. Costco uses Frontiers and Noritsus and makes profiles available. Most high end labs will give you a profile (otherwise, you are being ripped off). Then you can use soft proofing to detect and map out of gamut colors. If you are in aRGB and soft proof to see what colors are clipped in sRGB, you still do not know what colors that fit in sRGB would be clipped in printing. Also, if you have a profile with perceptual rendering (such rendering is not available for matrix profiles such as sRGB, aRGB, and ProPhotoRGB) you could try that to map the out of gamut colors.
The most recent inkjets have a considerably wider gamut than photo paper, and the use of ProPhotoRGB is recommended for use with these.

Help: Need to finding Best way of Processing XML feeds into MySQL

Similar Messages

Maybe you are looking for