Knowledge Base

I- BASIC PRINCIPLE

PcVue’s proprietary archiving was not designed to interact with third-party tools. As a result, only PcVue can retrieve the information contained in such archiving units.

However, there are 2 proprietary archiving formats:
-> ASCII, which can be read using a text editor
-> BINARY, which can only be read by PcVue.

To migrate a proprietary archive into an HDS archive, the principle is as follows:
1) Creation of the target unit (archive unit, tables)
2) Extraction of packets of historical data
3) Generate files in HDS import format
4) Use the “DBImport” utility to inject logs into HDS tables

II- CREATING A DATABASE ARCHIVE

This chapter outlines the steps to be followed when creating an HDS trend for a variable initially archived in proprietary format.

Step1: Database declaration, i.e. at least
– a database
– a trend table
– a trend group

Step 2. Creation of a database archiving unit

Step 3. Variable trending

Step 4. Handling the “Primary” option.

Please note! The proprietary trend MUST be primary for extraction.

III- EXTRACTION

III.1- Extraction tool for proprietary ASCII archives:

When archives are in ASCII format, a tool is available to extract data directly from files

directly from the files, bypassing PcVue and generating HDS import files.

This tool only supports proprietary ASCII archiving units whose date format is expressed in expressed in UTC with the following syntax: yyyyMMddThhmmss.lllZ

How it works :
1) each “data” file in the source directory is read
2) trend and log tags are interpreted
3) a trend export file is generated if at least one trend exists
4) an export file for logging is generated if at least one logged event exists, and specified by the input parameters.
Please note: the list separator must be “ ; ” on most French OS.

1. Insert the path for the varexp file
   Note: the varexp.dat file path is only required for exporting alarm logs.
2. Select “Load” to load the file
3. Source folder: insert the source folder where the proprietary unit is located
4. Target folder: Select the target where the file convert should be saved
5. Convert: Select convert to start the conversion

III.2- Proprietary archives in any format:

Whatever the format of the proprietary archive, PcVue itself is able to extract the data.

This chapter describes the end-to-end procedure for implementing this solution.

III.2.1. Setting up the library
The file supplied is a PcVue library which will need to be integrated into the project with access to the :

Step 1. Unzip and place the library in the project’s local user libraries.

Step 2. From the PcVue project, declare this library :

III.2.2.Export procedure

III.2.2.1.Description

The procedure uses the SCADA Basic “SVTREND” instruction to retrieve archived variable data from buffers, regardless of the read-accessible archiving format.

The procedure exports all trending variables declared in the project’s C\HISTO.dat file.

Exports take place between a start date and the current date. The start date can be configured via the @EXPORT_TO_HDSFILES.ConfigExport.StartDate variable (format DD/MM.YYYY).

As data extraction may take more or less time, it is possible to configure an extraction step expressed in days. Variable : @EXPORT_TO_HDSFILES.ConfigExport.ProcessStepInDays

Once extracted, the data is written to a CSV file in the format expected by the HDS tables. Depending on SQL Server regional options, decimal and list separators may change. The @EXPORT_TO_HDSFILES.ConfigExport.DecimalSeparator and @EXPORT_TO_HDSFILES.ConfigExport.ListSeparator variables can be used to adapt the export.

Note: maximum number of variables to export = 4000

Below is the list of internal variables:

III.2.2.2.Start export

Step 1. Open the “ExportSystemMimic” screen of the installed library:

The synoptic allows you to:
[1] Load SCADA Basic programs (“LOAD”)
[2] Create internal system variables for export (“IMPORT”)
[3] Modify export configuration (“GET CONFIGURATION”)
[4] Start and stop the export procedure
[5] Access the export directory and launch the “DB Import” tool

Step 2. To trigger the export, follow the steps below:
a) Click on “Load” to load the SCADA Basic programs
– L_EXPORT_FOR_HDS/CONFIGURATION.SCB
– L_EXPORT_FOR_HDS/EXPORT_TRENDS.SCB 

b) Click on “Import” to create the variables.

The procedure triggers an XML import that can be deleted from the Smart Generator.

c) Relaunch the “ExportSystemMimic” screen: variables are displayed in the grid.
d) Set import parameters (ConfigExport branch)
e) Click on “Get Configuration” to load the configuration
f) Click on “Start” to launch export
Variables are exported one after the other to the destination directory

Step 3. Click on “Open folder” to check the arrival of the files.

III.2.3.Notes

• Exporting can be a lengthy process, depending on the depth of the archive.
•  SCADA Basic has an infinite loop threshold which can be used to stop any program whose execution time exceeds this threshold.

Although the SVTREND instruction for retrieving archives is asynchronous, the processing of result buffers is synchronous. You must therefore ensure that the processing of a request does not exceed the threshold defined in the project (default: 30 seconds).

If necessary, you can use the parameter of the variable : @EXPORT_TO_HDSFILES.ConfigExport.ProcessStepInDays.

IV- IMPORT

DBImport can be used to inject exported files into HDS tables in the following format (for trends) :

Notes:

1) The list separator and decimal separator may vary according to OS regional options.
2) Export files must have a “.dat” extension to be taken into account by “DBImport”.

IV.1. Procedure

Step 1. Click on “Start DBImport” to launch the utility (or via the start menu).

1. Enter export directory and start/end dates (make sure start date is earlier than export start date)

b. Select the variables to be imported

c. Set destination table connection parameters

d. Start import

IV.2. Check

Step 1. From the PcVue project, toggle the “primary” trends and check.

Step 2. From SQL Server, the PcVue project toggles “primary” trends and checks.

IV.3. Known problems

Depending on the regional options of the SQL Server machine, the import may encounter an error related to the decimal separator used.

The decimal separator used by PcVue is the “.”, so the files will contain this separator:

To overcome this problem, we can :

1. either modify the export configuration via the variable : @EXPORT_TO_HDSFILES.ConfigExport.DecimalSeparator

b. Either modify regional options or directly in SQL Server.

Download :

Library : L_EXPORT_FOR_HDS

Converter executable : Prop2HdsConverter