Citadel database missing trace

Once again we have a problem.
Client is running 6.1 (Build 27). Missing data from a few objects for 2 and a half months! Suddenly reappears. No machine down tme, no errors, just gone. Tried to rebuild database, no luck.
This is becoming a problem, as much as it would pain me to move to a new software platform we may have to at this point. Constant database corruptions cannot happen when the clients need these reports for internal and other health type agencies.
Citadel is a platform that simply is not reliable. 6.2 show the same errors as well.
Example problem. Tanks record current, high and low values that are trended. The low and high disappear but current stays. And all of the points for the pump (control, monitor and status) are gone ont he same dates. Then suddenly reappear later with no system restarts at all.
EDIT: We know the system was working, we log onto spreadsheets as well, the data is correct in there. But the report system is automated in lookout, not an excel type program.
Thanks, again,
Mike
Message Edited by Mike@DTSI on 06-19-2009 01:27 PM
Mike Crabtree - Lead Developer
Destek of Nevada, Inc. / Digital Telemetry Systems, Inc.
(866) 964-6948 / (760) 247-9512

Got some screenshots
1 - This is one week view from today (07/02-07/08). The green line (the logical connection for the Control Bit (\\.\ServerWell4\BC)) shows data as being in the database for the hypertrend.(The gold and dark red lines are other wells, which have the same issues.)
2. This is the SQL pull using the datatable object. Pulls from the Citadel database. Looking at the days 07/01-07/08 it shows 0 for everything, but as you see in the last pic, hypertrend sees the data. The SQL string:
SELECT MATH_ETM(TO_DISCRETE("TCWDSCADA/Server/Well4/BC")), MATH_starts(TO_DISCRETE("TCWDSCADA/Server/Well4/BC")), MATH_stops(TO_DISCRETE("TCWDSCADA/Server/Well4/BC")), MATH_ETM(TO_DISCRETE("TCWDSCADA/Server/Well4/BM")), MATH_starts(TO_DISCRETE("TCWDSCADA/Server/Well4/BM")), MATH_stops(TO_DISCRETE("TCWDSCADA/Server/Well4/BM")), MATH_etm(TO_DISCRETE("TCWDSCADA/Server/Well4/BS")), MATH_starts(TO_DISCRETE("TCWDSCADA/Server/Well4/BS")), MATH_stops(TO_DISCRETE("TCWDSCADA/Server/Well4/BS")) FROM IntData WHERE LocalTime BETWEEN '2009-7-2' AND '2009-8-2' and IntInterval ='1'
BC=Control, BM=Monitor, BS=Status. The reason we pull the LocalTime from 2 days into month and after the month is because the citadel database returns values from the days previous. Its wierd stuff.
July shows all 0. April has 2 days (04/01-04/02)
Well 4 in this example runs everyday. weekends, weekdays, everyday.
DBFolderT and SysfolderT have no remote connections except for the remote property is linked to a datatable field. the datatable is only loaded on system start, or manually which doesnt happen for this system.
Thanks again
Mike
Mike Crabtree - Lead Developer
Destek of Nevada, Inc. / Digital Telemetry Systems, Inc.
(866) 964-6948 / (760) 247-9512

Similar Messages

Logging large .ctl to citadel database is slow

Hi, All
I'm using LV 8.5 with DSC.
I'm logging my own.ctl types shared variables to citadel database. This own.ctl includes 52 piece doubles, strings and boolean datatypes. I have 200 piece this tape of shared variables and I record all that shared variable information to citadel database. All operations as writing, reading, achieving, and deleting database are very slow. They also consume lot CPU time. Is there any capacity limits on citadel database ? Which is best structure to record this kind of data to database? I have tested (see attacment) structure but it seems to be very slow because I have to delete and create database trace references after 1st database is full.
BR,
Attachments:
code_structure.jpg ‏51 KB

When you say "...to Log Data only when necessary... " I assume you are using Set Tag Attribute.vi to establish this behavior.
National Instruments recently added a feature (as a hot-fix) to the DSC Engine to be able to ignore timestamps coming from servers to prevent to log values with "back-in-time" timestamps. Citadel is really critical taking values back in time (logs a ) and therefore retrieval of Citadel data with such back-in-time traces could act wired.
You can find more info from:
Why Do I See a Lot of NaN (Not-a-Number) In My Citadel Database When I Use the Set Tag Attribute.vi?
blic.nsf/websearch/B871D05A1A4742FA86256C70006BBE00?OpenDocument>How Do I Avoid Out-of-Synch (a.k.a....
The Hot-Fix can be found:
LabVIEW Datalogging and Supervisory Control Module Version 6.1 for Windows 2000/95/98/ME/NT/XP -- Fi...
I assume you run into such a use case - maybe. It happen to me, too . And I've created a small VI which would analyze traces on back-in-time (NaN - Not a number) values. I assume the missing Data in DIAdem are those Not-a-Numbers aka Break.
If you still encounter some problems after applying the DSCEngine.ini UseServerTimestamps=false, you might contact a National Instruments Support Engineer.
Hope this helps
Roland
Attachments:
BackInTimeAnalyzer.llb ‏622 KB

CITADEL DATABASE NOT CONFIGURED AS A RELATIONAL DATABASE

HI:
I have the same problem described by another member before, and I haven't found any resolution of this problem:
The problem was:
>I enabled database logging, and configured the shared variables that I wanted to log. Then I deployed the variables. When I try to read alarms and events (Alarm & Event Query.vi), I get this message:
>Error -1967386611 occurred at HIST_RunAlarmQueryCORE.vi,
Citadel: (Hex 0x8ABC100D) The given Citadel database is not currently
configured to log alarms to a relational database.
Please help me with this topic
Thanks in advance

I'm configuring the variables to be logged in the same way that appears on the file you send, but it doesn't work... I don't know what else to do.
I'm sending you the configuration image file, the error message image and a simple vi that creates the database; after, values are logged; I generate several values for the variable, values that are above the HI limit of the acceptance value (previously configured) so alarms are generated. When I push the button STOP, the system stops logging values to the database and performs a query to the alarms database, and the corresponding error is generated... (file attached)
The result: With the aid of MAXThe data is logged correctly on the DATA database (I can view the trace), but the alarm generated is not logged on the alarms database created programatically...
The same vi is used but creating another database manually with the aid of MAX and configuring the library to log the alarms to that database.... same result
I try this sabe conditions on three different PCs with the same result, and I try to reinstall LabVIEW (development and DSC) completelly (uff!) and still doesn't work... ¿what else can I do?
I'd appreciate very much your help.
Ignacio
Attachments:
error.jpg ‏56 KB
test_db.vi ‏38 KB
config.jpg ‏150 KB

Latency in accessing Citadel Database over the LAN

Hi!
I have two systems running LabVIEW 2013 with DSC 2013 and I am using one of the systems to acquire (using shared variables) and log data internally in the citadel database. I have made an application to visualize this data either locally on the same PC or over the LAN. When I am using the application on the same local machine which is logging the data, its all working fine and very fast. But when I access the same database over the LAN, the application takes a finite amount of time (anything between 2-18 secs everytime) to retrieve and show the data. Since the database size is very small (less than 5MB), logging rate is low (1Hz), number of traces are not much (14 traces), data is boolean, it should not take so much time to show the values over the LAN (the application should be as snappy as it is on the local PC). I am at my wits' end but I am not able to reduce this latency or find any logical reason for it to be present.
I specify the logging PC's name whenever I want to retrieve the trace (as shown in the attached picture), I have a feeling that if I replace that with the IP address of the system, things should become fast. But whenever I try this, LabVIEW throws up an error saying that the specified trace/database does not exist.
I am attaching a pic of the architecture and a simplified image of how I am retrieving the data from the database with this post.
Any help shall be deeply appreciated.
Attachments:
LAN Architecture.png ‏23 KB
Accessing database.png ‏14 KB

To enable external access without VPN, you must assign an external IP address and also map that external IP to the server. Additionally, you must open the associated ports (http) on the firewall (doesnt matter what firewall you use). You will have to map the extranal ip to the internal IP that you have set on the EBS server configuration. In some cases, you have to enable the proxy as well.

How do you change values after logging data to the citadel database (DSC)?

How would one go about changing values that already exist in the citadel database. So say I were logging a numeric value with a timestamp for 1 year and then wanted to change a few values for the month of June because my sensor wasn't working at that time. How could I do this? Is it possible without re-creating the whole database over?
Thanks
Matt

mattyk wrote:
Hey Ben,
Thanks for the quick response. I called NI and asked about this because this would be a big show stopper for me. Their applications engineer said that the best way to change previous data would be to read in the trace that you want to change, modify it and save it as a new trace. I am also looking into SQL querying. I was able to connect to the citadel database by setting up a ODBC data source and using the database connectivity toolkit to query the database. This worked. Now I am trying to come up with a statement in SQL to change previously saved data. I'll Keep you posted.
Thanks
Matt
That reinforces my suspicions.
I have an app running for about 8 years now that keeps a paper trail to enusr ethe cartdiges used in respirators (operated NIOSH National Institute of Occupational Safety and Health par teh CDC) that use DSC to enusre nobody can ever falsify those records. If they can be hacked I my customer and anyone using a respirator with a NIOSH sticker on the side of it, want to know.
Ben
Ben Rayner
I am currently active on.. MainStream Preppers
Rayner's Ridge is under construction

Editing tag values logged in tag history / citadel database

Hi,
I having been logging data from input/output tags and memory tags to my citadel database over the past week and due to some errors in my vi (my fault) some values are showing on my trends that are incorrect. These values are affecting my calculations (e.g. zeros, infinite numbers etc) and I would really like to delete them if possible.
Does anyone know a way of going in to the database/tag history and selecting and deleting values whilst still leaving other data in tact?
Many thanks,
Stuart

You can remove the data from a Citadel database using LabVIEW, Lookout or Measurement and Automation Explorer (MAX). This is a two step process:
First you archive the data that you want to delete. When archiving, choose to delete this data from the original database. The archiving operation will move the data from your database to new database.
The second step is to delete the new archived database. This removes the extra database that was created by the archive operation. Once the data has been archived you can remove the archived data by either deleting the database manually or by using file VIs from LabVIEW.
The document elaborates this process for MAX, LabVIEW and Lookout separately
Measurement and Automation Explorer (MAX) You must have Historical Data Viewer installed. It comes with Lookout 5.0 and LabVIEW DSC 6.1.
To remove the historical data in MAX, follow these steps:
Under the Historical Data folder in MAX, select the database that you want to remove data from.
Right-click the database and select New View»Trace. This creates a trace view with the default name of New View under the database folder.
On the Trace Attributes page, click on Add new traces. Select the traces you want to archive.
Select a starting and stopping time.
Right-click New View and select Archive.
Enter a directory path to store the archived data.
Check the Delete data from source database checkbox to delete the data from the source database.
Click Start.
Once it is complete, click Close.

Citadel database error 81bc0163 (Citadel 4.3)

I’m not sure how my database was corrupted (I suspect a NaN)
but the result is that when displaying a certain period of time, either from
within the LabView DSC module or with the Measurement and Automation Explorer (MAX),
the LabView DSC (or MAX) crashes by exiting without notice. If you attempt to
archive the data, you get the error 81bc0163 and archiving is aborted. How to
solve:
1. Make a complete copy of the database directory (with the
.thd and other files). You will need the dates on which the .thd files were
created, and the Citadel database engine wants to repack these files and thus
change the dates (Citadel will sometimes use ‘holes’ in existing files to store
new data, but it generally creates files and stores data sequentially.)
2. Using MAX, open your copied Citadel database and create a
new view to it. Include the 1st 20 traces. Plot these traces (select
them and then click on the Display tab) for the entire duration (this can take
a while). If MAX does not crash, plot the next 20 traces, until MAX crashes. The
goal is to find which trace cause MAX to crash. (If you can plot all traces for
all times, you don’t have the problem I encountered.)
3. Now create a view that includes the 20 twenty traces that
caused the crash and Display it for various periods of time . Select an end
date, and then choose the Display tab. If MAX crashes, then repeat with a
different end date, util you can isolate the date and time of the faulty
record. Be sure to use a very narrow Display time to get good resolution. A
binary search of time periods is the most efficient, unless you already have
some clue about the problem date.
4. Once you have identified the date and time of the date
that causes the crash, exit MAX and then delete the .thd file whose creation
date is just prior to the chars date and time.
5. Restart MAX and open the view of your copied data base.
BE PATIENT, it can take many, many minutes for the Citadel database to deal
with the deleted .thd file, but it appears that the Citadel database is robust
enough to recreate its indices.
6. Create a second new view of the copied database (this resets
the time from the beginning of creation to the distant future), select the
critical 20 traces, and click on the Display tab to make sure the database can
now be viewed without crashing. If, not repeat the process with yet another
copie database.
7. Now, rename the copied and the original folders so that the
fixed database replaces the original. Verify that the ‘new original’ is OK by
running MAX and crating a view of the 20 traces and displaying them for all
times.

I forgot to include the full stack trace:
java.lang.UnsatisfiedLinkError: /nfsmounts/risc14/cmsd/gdfr/dbxml/dbxml-2.2.13/install/lib/libdb_java-4.3.so: /nfsmounts/risc14/cmsd/gdfr/dbxml/dbxml-2.2.13/install/lib/libdb_java-4.3.so: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1751)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1676)
at java.lang.Runtime.loadLibrary0(Runtime.java:822)
at java.lang.System.loadLibrary(System.java:992)
at com.sleepycat.db.internal.db_javaJNI.<clinit>(db_javaJNI.java:48)
at com.sleepycat.db.internal.DbEnv.<init>(DbEnv.java:200)
at com.sleepycat.db.EnvironmentConfig.createEnvironment(EnvironmentConfig.java:738)
at com.sleepycat.db.EnvironmentConfig.openEnvironment(EnvironmentConfig.java:691)
at com.sleepycat.db.Environment.<init>(Environment.java:30)
at org.oclc.da.gdfr.registryprototype.pvt.CollectionDbXml.createDefaultManager(CollectionDbXml.java:361)
at org.oclc.da.gdfr.registryprototype.pvt.test.CollectionDbXmlTest.setUp(CollectionDbXmlTest.java:324)
at org.oclc.da.gdfr.registryprototype.pvt.test.AllTests.main(AllTests.java:22)
I'm also installing the DB with this tar:
dbxml-2.2.13.tar.gz

I can not open an citadel database whit diadem

i can not open an citadel database whit diadem, the diadem shown error number 104 , "unable to connect to specified server . what have i to do ?

Hi Ramatajama,
Most likely you are just missing one simple step, which is to turn on the Citadel service that allows any client, such as DIAdem, to get to the data in the Citadel database.
You do not have to have the Citadel database on the same computer that DIAdem resides on, but you DO have to go to "NI Service Manager" on the DIAdem computer and "start all services". Usually the NI Service Manager icon shows up in your system tray at the far bottom right of your computer screen.
Ask again if this isn't enough info,
Brad Turpin

Lookout 6.2 loading buffered data from PLC into Citadel database?

We are testing a Lookout small process application <200 I/O that is designed for continuous application. This is not a critical process, therefore we don't want to commit to duplicate Lookout Systems. However, we would like to buffer 24 words(16bit) in the PLC for 5-10 minutes so we would not loose that data when the computer running Lookout does a restart for example.
Our problem is how do we get this data into LookOut Citadel database in the proper time frame.
If loading this data into Citadel is not possible with the original trace, can we load into another database file and view this data separately?
Any suggestions would be appreciated!
Solved!
Go to Solution.

First, you need to be able to get the data as well as the timestamp from the PLC.
With all the data and timestamp, you can use the Logger object to log data to Citadel. Take a look at this example. The Logger_discrete object.
http://zone.ni.com/devzone/cda/epd/p/id/3816
But the problem is that the data is not logged to the original trace. It is logged to the logger object trace.
Ryan Shi
National Instruments

Is there a way to test for integrity of the Citadel Database?

Gday,
Currently I am logging OPC data to a Citadel 5 database (Labview Version 8.2.1).
I have done some integrity testing to try and "break" the citadel database and I noticed that if you delete any of the .cdpg files no alarms are flagged or notifications given in MAX or elsewhere to show that the database has been comprimised.
The result of deleting or the corruption of any of these .cdpg files will result in valid data being 'moved' back in time as the citadel databases only log changes in value with no corressponding timestamp. For example if you delete from 9.00am to 10.00am then all data logged after 10.00am will now start at 9.00am.
So I was wondering if there is anyway to either correct or even test for corruption in a citadel 5 database.
Regards
Ben

This should not happen according to the design of Citadel. Also I tested with current Citadel, and didn't see this problem. I'm not sure if it was a bug in the old Citadel(the one with DSC 8.2), but for current Citadel, I can't reproduce it.
There are many files in database folder. The .cdpg file is the data file, containing all the logged data. Some other files, such as .cdin, .cdih, .cdib, are the index files. When you read data from a trace, Lookout will first find the index from those files, and then locate the position of the data in .cdpg files. Actually Citadel will rebuild the index files if it finds something changed or damaged. If you delete one .cdpg file, Citadel will rebuild the index files. With the new index files, you can work with the database without any problem. The only difference is that the data in the deleted file can't be viewed then, just like the data was nevered logged. So, to delete the .cdpg file won't damage the whole database. The new data will be logged to database with the real time or the user-defined time. Citadel does log the delta value, but it also records the time for each trace. The detailed architecture is too complex to describe.
Can you give some screenshots to show the problem?
Ryan Shi
National Instruments

How to import measurment data from a Excel File to Citadel DataBase?

A Microsoft Excel File is being created in a PC and I want to include that flie´s data into my citadel Database wich is in another PC.

I give a example "Importing Spreadsheet Data to the Citadel Database" in the LabVIEW Version 6.1 and LV-DSC 6.1.
LabVIEW 7.0 with LV-DSC you must in the ini file "DSCEngine.ini"
set NoShutdownLog=1 and UseServerTimestamps=TRUE.
kind regards
amcholger
Attachments:
Write_File_to_Citadel_Example.zip ‏139 KB
Write_File_to_Citadel_VI_Server.zip ‏289 KB

How to add an array of data in Citadel database

I have an array of data which I want to directly put it in the Citadel Database. Normally the write tag VI writes one value at a time. How can this be done ?

It depends on what you really want. Could you be a little more specific? Does each point have its own separate timestamp? If so, you could simply put the "Write Tag.VI" inside a FOR LOOP and insert values one after another. This would give a separate timestamp to each value. I have attached a simple VI that shows how to do this.
Or are you looking to record some type of vector, where you log a complete array of data at each interval? I don't think this is possible (I'm using DSC 6.02) unless the data-type is a bit-array, in which case you would simply use "Write Data (bit array).vi".
Another option, depending on what you are aiming for, would be to create your own "VI-based Device Server" and use the VIs on the palette submenu call
ed "DSC Server Development". With your own Device Driver you can write multiple datapoints to the input queue simultaneuosly. In this way, you can write more than one point at a time... but that doesn't mean that each time interval will contain an array of data. Assuming that the data is analog, the data will still be logged to Citadel "one after another" so that when you look at it with the HTV, you will see one curve of analog values.
Attachments:
Write_array_to_citadel.vi ‏17 KB

Write to Citadel database from LabVIEW?

Hi. I'm a newbie when it comes to Lookout and the Citadel database... so here's my question for you all:
We have various labview applications that log data from instruments. We would like to be able to take this data and store it as "tags" in a Citadel database.
What do you think is the best way to do this?
There are ODBC drivers for the Citadel database, but they appear to only allow you to query the database to retrieve values... you can't write data to the database (so its read-only).
Are they any "Lookout VIs" or Citadel VIs that can write to the Citadel DB from LabVIEW? Or a DLL or something?
Any ideas?
regards,
John Paul

Very nice question John Paul. I take it you already have Lookout and LabVIEW. In this case, with the latest versions of either, simply use the LabVIEW datasocket to publish the items on your front panel. Lookout then can subscribe to these once you register the LabVIEW computer. Once you have subscribed to them in Lookout, you are then able to set up logging. You may need to create an expression object to route the data to citadel.
If you just want to use LabVIEW and the Citadel database, you might try VI Logger. This works in our configuration utility - MAX, and also comes with a set of VIs for LabVIEW. One caveot, the Citadel database in VI logger is newer (and faster) than the Citadel database in Lookout. I believe NI is working to synchronize
the version in the future, and its hard to say when. So till then, you would not be able to merge data in the dislike citadel databases. However, you can always use and ODBC call set to extract from both and store in a more permanent SQL database such as access, oracle, ms back office, etc.
Good Luck,
Preston Johnson
Business Development Manager
Industrial Applications
National Instruments
Preston Johnson
Principal Sales Engineer
Condition Monitoring Systems
Vibration Analyst III - www.vibinst.org, www.mobiusinstitute.com
National Instruments
[email protected]
www.ni.com/mcm
www.ni.com/soundandvibration
www.ni.com/biganalogdata
512-683-5444

Archive citadel database in MAX hangs

I am trying to Archive a 4.13 GB uncompressed DSC database on LabVIEW 2013 SP1 (although the database was started with LabVIEW 2013). I have tried twice and it gets stuck (new location folder size is at 151 MB both times). The progress bar says "Archiving database=20.5%" and "Copying Data=83.0%". I have chosen the option "Destroy source data after it has been archived (local computer only)". The status says "copying data" and I no timestamps of files in the new folder have changed after about 9 hours of waiting. If I cancel, MAX cannot display any Citadel databases. This is fixed if i reboot.
Any ideas?

Hello barkeram,
Where are you storing the archive (locally or remote)?
Are you following the steps in one of the following documents?
http://digital.ni.com/public.nsf/allkb/E076A0661E03F1EB862571A800079E7B
http://digital.ni.com/public.nsf/allkb/2B0C74744BB37391862571F500067C64
Can you try to navigate to the file path of the database and duplicate the file? Afterward, manually add the new database to MAX and try to Archive the new file.
Regards,
Thomas C.
Applications Engineer
National Instruments

Options to edit historical data in Citadel database (Lookout 6.0)

We are running a new installation of Lookout 6.0
I am looking for ways to edit the historical data that is found in the Citadel database. For example, if an alarm clears before I can enter comments, I would like to go to the database after the fact and enter comments regarding the alarm.
We are running this on XP pro operating system.
Thanks,
Alan

Hi Alan,
I am afraid this will be difficult, if at all possible. Citadel data, by design, can be written to (and edited?) only by the product using it, i.e., Lookout, LabVIEW-DSC, etc. Outside of these products we can only retrieve the data -- not edit or add to it. As you are probably aware, some of the options for writing user data (as opposed to IO/system data) from within these products is to use the Logger Object in Lookout and the VI-Server approach in DSC (http://zone.ni.com/devzone/conceptd.nsf/webmain/5a921a403438390f86256b9700809a53).
So, I guess one option is to "annotate" / write additional data separately using the Logger Object, referencing the Alarms somehow.
Having said that, I believe Lookout 6.x (Citadel5) uses MSDE for storing Alarm data (other data is still stored in the native Citadel database). You could explore this -- try opening the MSDE database from Query Analyzer, from instance, and see if it can be edited. I haven't tried this.
Hope this gives you some ideas.
-Khalid

Citadel database missing trace

Similar Messages

Maybe you are looking for