| Oracle® GoldenGate Veridata Administrator's Guide 11g Release 2 (11.2.1.0.0) Part Number E29092-01 |
|
|
PDF · Mobi · ePub |
| Oracle® GoldenGate Veridata Administrator's Guide 11g Release 2 (11.2.1.0.0) Part Number E29092-01 |
|
|
PDF · Mobi · ePub |
This chapter describes how to use Oracle GoldenGate Veridata. It provides an overview of roles and interactions of the components, how to configure components, and how Oracle GoldenGate Veridata compares tables.
This chapter includes the following sections:
Oracle GoldenGate Veridata compares one set of data to another and identifies data that is out of synchronization. Oracle GoldenGate Veridata supports high-volume, 24x7 replication environments where downtime to compare data sets is not an option. By accounting for data that is being replicated while a comparison takes place, Oracle GoldenGate Veridata can run concurrently with data transactions and replication, while still producing an accurate comparison report.
Oracle GoldenGate Veridata will map column data types across different types of databases automatically, or you can map columns manually in cases where the automatic mapping is not sufficient to accommodate format differences in a heterogeneous environment. For detailed information about this feature, see the online help.
For the purposes of this documentation, the following terms are considered synonymous:
tables and files
columns and fields
rows and records
Oracle GoldenGate Veridata Server
The Oracle GoldenGate Veridata Server performs the following functions:
Coordinate the execution of Oracle GoldenGate Veridata tasks
Sort rows (optional)
Compare data
Confirm out-of-sync data
Produce a report for review
Oracle GoldenGate Veridata Web User Interface
Oracle GoldenGate Veridata Web User Interface (UI) is a browser-based graphical user interface for these activities:
Configure comparison objects and rules
Initiate comparisons
Review the status and output of comparisons
Review out-of-sync data
Oracle GoldenGate Veridata Repository
The Oracle GoldenGate Veridata repository is a collection of database objects that persists configuration information to disk, saving it permanently as a user environment.
Oracle GoldenGate Veridata Agent
The Oracle GoldenGate Veridata Agent executes the following database-related requests on behalf of the Oracle GoldenGate Veridata Server:
Fetch and return blocks of rows to compare
Return column-level detail for out-of-sync rows
Oracle GoldenGate Veridata Manager
The Manager process is part of the C-code based agent that is required for the NonStop platform and optional for the Oracle database. It controls the Oracle GoldenGate Veridata Agent process. A Manager is not used in a Java agent, which is used for the other databases that are supported by Oracle GoldenGate Veridata.
Oracle GoldenGate Veridata CLI (Command Line Interface)
Oracle GoldenGate Veridata includes a command line client to Oracle GoldenGate Veridata Server. The program name is vericom. This interface enables you to run comparisons, specify overrides, and stop Oracle GoldenGate Veridata Server by using automated programs. Configuration and monitoring functions must be performed through Oracle GoldenGate Veridata Web.
This section explains how to configure the objects that are to be compared and how Oracle GoldenGate Veridata processes comparisons.
To begin using Oracle GoldenGate Veridata, you need to create some objects that help you identify the data you want to compare and help you manage your work.
Configure data source connections: Oracle GoldenGate Veridata Server connects to Oracle GoldenGate Veridata Agents that interact with the databases that contain the data that is to be compared. A connection is defined by a host, the port number of an Oracle GoldenGate Veridata Agent on that host, and the data source that is accessed by the agent.
Configure groups: You need to configure at least one compare group that is linked to a set of source and target data source connections. A group is a logical container for organizing the objects to be compared.
Configure compare pairs: You need to configure one or more compare pairs for each group that you create. A compare pair is a set of corresponding source and target tables or files. As part of configuring compare pairs, you map source and target columns to establish a structural relationship between the two objects.
Configure profiles: A profile contains settings for run-time parameters and can be applied globally to a job, as well as to a specific compare pair as an override to the job profile. Profile parameters encompass considerations such as sorting method, thread and memory usage, reporting output, and so forth. Defining run profiles is optional, because Oracle GoldenGate Veridata includes a default profile that contains settings that apply to most usage scenarios. However, as you gain experience with Oracle GoldenGate Veridata, you may want to customize the default profile or create your own custom profiles.
Configure jobs: A job is a logical container for one or more compare groups and is the unit of work by which comparison processing is executed. Within one or more jobs, you can manage and run large volumes of compare groups across numerous databases and systems, and you can control the timing of those comparisons.
For more information, see the Oracle GoldenGate Veridata online help.
Oracle GoldenGate Veridata relies on some form of unique identifier to order rows for comparison.
Primary Key: By default, Oracle GoldenGate Veridata uses the primary key if one is available.
Unique Key: If no primary key is defined, Oracle GoldenGate Veridata uses the smallest unique index
User-defined Key: If a table or file has neither a primary nor unique key, you can define an existing index or set of columns for comparison purposes when defining a compare pair. However, although primary or unique keys can be mapped automatically, user-defined keys must be mapped manually. A user-defined key can also be used to override existing keys or indexes if you prefer a different ordering method.
For more information about choosing and mapping keys for comparison, see the Oracle GoldenGate Veridata online help.
Comparison activities are divided into the following steps. You can change some of the aspects of these steps by making parameter changes in Oracle GoldenGate Veridata Web.
In the initial comparison (or row hash) step, rows are retrieved from the source and target tables with a query. If the source and target databases are of different types, the columns are converted to a standardized data type format for accurate comparison. By default, Oracle GoldenGate Veridata compares rows by comparing all columns of the primary key literally (value-for-value) and by using a hash value for all non-key columns. The unique digital signature that is used to calculate the hash value shrinks the data to be transferred over the network for the comparison, while still providing a highly reliable (but not absolute) and efficient mechanism for determining whether two rows contain the same or different column values.
For more assurance of discovering out-of-sync rows, you can configure Oracle GoldenGate Veridata to compare non-key rows column-for-column, instead of using a hash. Full-column comparisons reduce the processing performance in proportion to the number of columns, and they increase network usage.
On the NonStop platform, you can use the delta processing feature during the initial comparison step if you are using server-side sorting. Delta processing is a performance feature by which Oracle GoldenGate Veridata detects which data blocks in the database were modified since a previous comparison and only compares the rows in those blocks. Rows in unchanged blocks are skipped. The default is to compare all rows regardless of whether they changed or not.
There are two steps to delta processing:
Collecting the base modification time of the previous run for subsequent delta comparisons. This step is always done when delta processing is enabled for a compare pair.
Comparing data that has been modified since the base comparison, using the information that was collected in the first step. This step is enabled by clicking the Enable Delta Processing button on the Compare Pair Configuration page and the Run/Execute Job page of Oracle GoldenGate Veridata Web. The Disable Delta Processing button allows you to disable the delta comparison step in case there were modifications, such as table reorganizations, that can invalidate the collected delta base information.
For more information about delta processing, see the Oracle GoldenGate Veridata online help.
After the initial comparison, rows that appear to be out-of-sync are stored in a maybe out-of-sync (MOOS) queue in memory, because at this point the comparison is inconclusive. When replication is working concurrently with a comparison, especially if there is replication latency, rows can appear to be out-of-sync when, in fact, the current data is in flight (somewhere in the replication flow) and replication will soon synchronize them again.
The confirmation, or confirm-out-of-sync (COOS), step ensures accurate results by confirming row status in a changing environment. In this step, rows are extracted from the MOOS queue, and their status is evaluated as one of the following:
in-flight: the row was out-of-sync in the initial comparison step, but has since been updated. In this case, it is assumed that replication or another mechanism applied the change, but Oracle GoldenGate Veridata was unable to confirm that the rows were in-sync.
in-sync: the source row values were applied to the target row by replication or another method. Even a status of in-sync does not guarantee that the rows are synchronized at any particular moment if the underlying tables are continuously changing, but it does indicate that replication is working.
persistently out-of-sync: the row has not been updated since the initial comparison step took place, and therefore can be assumed to be out-of-sync.
By default, confirmation processing occurs in a thread that is parallel to the initial comparison step, but the confirmation of each row waits until after a specified replication latency threshold has expired. For example, if latency is 60 seconds, and the initial comparison step revealed an out-of-sync row at 9:30, then the confirmation step for that row is not performed until 9:31 to allow replication to apply any change that was in-flight. After latency is accounted for, rows can be confirmed as persistently out-of-sync and are stored in one or more out-of-sync reports.
Upon completion of a job, you can view the comparison reports and the out-of-sync report by using Oracle GoldenGate Veridata Web or by viewing the files themselves.
You have the option to store an out-of-sync report in binary format, in XML format, or both (or none).
OOS file: When stored in binary form, the OOS report is compatible with the Oracle GoldenGate Veridata programs. As such, it can be viewed from the Oracle GoldenGate Veridata Web interface, and it also can be used as the basis for re-comparing rows that were out of synchronization. To re-compare rows, you select run options to execute another confirmation step, which compares the current state of just those rows and then reports which ones remain out-of-sync after replication or another restorative procedure has been applied.
OOSXML file: When stored as XML, the OOS report is written to an OOSXML file and is stored in a structured way that conforms to an internal XML schema. XML has many advantages, the largest being that it can be manipulated easily by many tools. In its XML form, the file contains all of the information, including metadata, that is needed to select rows for resynchronization by external programs.
Each finished job, group, and compare pair generates a comparison report. The report file contains details about the comparisons that were performed, such as:
Comparison parameters used
The number of rows compared and out-of-sync
The timing of the comparison
Source and target data values
The files themselves are stored as follows:
The OOS files are located in sub-directories of the Oracle GoldenGate Veridata Server installation directory:
OOS files: shared/data/oos
OOSXML files: shared/data/oosxml
These directories are further organized by run ID, job name, group name, and compare pair. In the OOSXML directory, the files with the .oosxml extension are the control files. The files with sequential file extensions are the OOSXML chunks. The XML data is spread into multiple files (called "chunks") for performance purposes.
|
 Copyright © 2005, 2012, Oracle and/or its affiliates. All rights reserved. Legal Notices |
|
