Contents

List of Examples

List of Figures

List of Tables

Title and Copyright Information

Preface

• Audience
• Documentation Accessibility
• Related Documents
• Conventions

Part I Basic Administration

1 Monitoring

• Systems Monitoring: Breadth and Depth
• Monitoring Basics
  • Out-of-Box Monitoring
    • Metric Thresholds
  • Alerts
  • Notifications
    • Customizing Notifications
  • Corrective Actions
  • Blackouts
• Monitoring Templates
• User-Defined Metrics
• Accessing Monitoring Information

2 Enterprise Manager Security

• About Oracle Enterprise Manager Security
• Enterprise Manager Authentication
  • Repository-Based Authentication
  • Single Sign-On Based Authentication
    • Registering Enterprise Manager as a Partner Application
    • Removing Single Sign-On Configuration
    • Registering Single Sign-On Users as Enterprise Manager Administrators
    • Grid Control as a Single Sign-On Partner Application
    • Bypassing the Single Sign-On Logon Page
  • Enterprise User Security Based Authentication
    • Registering Enterprise Users as Enterprise Manager Users
• Enterprise Manager Authorization
  • Classes of Users
  • Privileges and Roles
    • Granting Privileges
• Configuring Security for Grid Control
  • About Enterprise Manager Framework Security
  • Overview of the Steps Required to Enable Enterprise Manager Framework Security
  • Enabling Security for the Oracle Management Service
    • Checking the Security Status
    • Creating a New Certificate Authority
    • Viewing the Security Status and OMS Port Information
    • Configuring Transparent Layer Security
  • Enabling Security for the Oracle Management Agent
  • Enabling Security with Multiple Management Service Installations
  • Restricting HTTP Access to the Management Service
  • Managing Agent Registration Passwords
    • Using the Grid Control Console to Manage Agent Registration Passwords
    • Using emctl to Add a New Agent Registration Password
  • Enabling Security with a Server Load Balancer
  • Enabling Security for the Management Repository Database
    • About Oracle Advanced Security and the sqlnet.ora Configuration File
    • Configuring the Management Service to Connect to a Secure Management Repository Database
    • Enabling Oracle Advanced Security for the Management Repository
    • Enabling Security for a Management Agent Monitoring a Secure Management Repository or Database
  • Configuring Third Party Certificates
    • Configuring Third Party Certificate for HTTPS Upload Virtual Host
    • Configuring Third Party Certificate for HTTPS WebTier Virtual Host
• Configuring Security for the Database Control
• Accessing Managed Targets
  • Credential Subsystem
    • Managing Credentials Using EMCLI
  • Pluggable Authentication Modules (PAM) Support for Hosts
    • Configuring PAM for RHEL4 Users
    • Configuring PAM for AIX Users
  • Sudo and PowerBroker Support
    • Creating a Privilege Delegation Setting
• Cryptographic Support
  • Configuring the emkey
  • emctl Commands
    • emctl status emkey
    • emctl config emkey -copy_to_credstore
    • emctl config emkey -copy_to_repos
    • emctl config emkey -copy_to_file_from_credstore
    • emctl config emkey -copy_to_file_from_repos
    • emctl config emkey -copy_to_credstore_from_file
    • emctl config emkey -copy_to_repos_from_file
    • emctl config emkey -remove_from_repos
  • Install and Upgrade Scenarios
    • Installing the Management Repository
    • Installing the First Oracle Management Service
    • Upgrading from 10.2 to 11.1
    • Recreating the Management Repository
• Setting Up the Auditing System for Enterprise Manager
  • Configuring the Enterprise Manager Audit System
  • Configuring the Audit Data Export Service
  • Updating the Audit Settings
  • Searching the Audit Data
  • List of Operations Audited
• Additional Security Considerations
  • Changing the SYSMAN and MGMT_VIEW Passwords
    • Changing the SYSMAN User Password
    • Changing the MGMT_VIEW User Password
  • Responding to Browser-Specific Security Certificate Alerts
    • Responding to the Internet Explorer Security Alert Dialog Box
    • Responding to the Netscape Navigator New Site Certificate Dialog Box
    • Preventing the Display of the Internet Explorer Security Information Dialog Box
  • Configuring Beacons to Monitor Web Applications Over HTTPS
  • Using ORACLE _HOME Credentials
  • Patching Oracle Homes When the User is Locked
  • Cloning Oracle Homes

3 Notifications

• Setting Up Notifications
  • Setting Up a Mail Server for Notifications
    • Setting Up Repeat Notifications
  • Setting Up E-mail for Yourself
    • Defining E-mail Addresses
    • Setting Up a Notification Schedule
    • Subscribe to Receive E-mail for Notification Rules
  • Setting Up E-mail for Other Administrators
  • E-mail Customization
    • E-mail Customization Reference
• Extending Notification Beyond E-mail
  • Custom Notification Methods Using Scripts and SNMP Traps
    • Adding a Notification Method based on an OS Command or Script
    • Adding a Notification Method Based on a PL/SQL Procedure
    • Adding a Notification Method Based on an SNMP Trap
• Passing Corrective Action Status Change Information
  • Passing Corrective Action Execution Status to an OS Command or Script
  • Passing Corrective Action Execution Status to a PLSQL Procedure
• Passing Job Execution Status Information
  • Passing Job Execution Status to a PLSQL Procedure
  • Passing Job Execution Status to an OS Command or Script
• Passing User-Defined Target Properties to Notification Methods
• Assigning Methods to Rules
• Assigning Rules to Methods
• Notification Coverage
• Management Information Base (MIB)
  • About MIBs
  • Reading the MIB Variable Descriptions
    • Variable Name
    • MIB Definition
• Troubleshooting Notifications
  • General Setup
  • Notification System Errors
  • Notification System Trace Messages
  • E-mail Errors
  • OS Command Errors
  • SNMP Trap Errors
  • PL/SQL Errors

4 User-Defined Metrics

• Extending Monitoring Capability
• Creating OS-Based User-Defined Metrics
  • Create Your OS Monitoring Script
    • Code to check the status of monitored objects
    • Code to return script results to Enterprise Manager
    • Script Runtime Environment
  • Register the Script as a User-Defined Metric
  • OS-Based User-Defined Metric Example
• Creating a SQL-Based User-Defined Metric
  • SQL-Based User-Defined Metric Examples
    • Example 1: Query Returning Tablespace Name and Percent Used
    • Example 2: Query Returning Segment Name/Type and Extent Count
    • Example 3: Embed a Long SQL statement in a PL/SQL Routine
• Notifications, Corrective Actions, and Monitoring Templates
  • Getting Notifications for User-Defined Metrics
  • Setting Corrective Actions for User-Defined Metrics
  • Deploying User-Defined Metrics Across Many Targets Using Monitoring Templates
  • Deleting User-Defined Metrics Across Many Targets Using Monitoring Templates
• Changing User-Defined Metric Credentials

5 Group Management

• Introduction to Groups
• Managing Groups
  • Group Home Page
  • Group Charts Page
  • Group Administration Page
  • Group Members Page
  • System Dashboard
• Out-of-Box Reports
• Redundancy Groups
• Privilege Propagating Groups
  • Creating Privilege Propagating Groups
  • Using the Group Administration Privilege
  • Adding Members to Privilege Propagating Groups
  • Converting Conventional Groups to Privilege Propagating Groups

6 Job System

• Job System Purpose and Overview
  • What Are Job Executions and Job Runs?
  • Operations on Job Executions and Job Runs
• Preliminary Considerations
  • Using Pre-defined Tasks
  • Creating Scripts
  • Sharing Job Responsibilities
  • Jobs and Groups
• Creating Jobs
  • Selecting a Job Type
  • Creating an OS Command Job
    • Specifying a Single Operation
    • Specifying a Script
    • Access Level Rules
  • Creating a SQL Script Job
    • Specifying Targets
    • Options for the Parameters Page
    • Specifying Host and Database Credentials
    • Returning Error Codes from SQL Script Jobs
  • Creating a Multi-task Job
    • Job Capabilities
    • Specifying Targets for a Multi-task Job
    • Adding Tasks to the Job
• Analyzing Job Activity

7 Starting and Stopping Enterprise Manager Components

• Controlling the Oracle Management Agent
  • Starting, Stopping, and Checking the Status of the Management Agent on UNIX
  • Starting and Stopping the Management Agent on Windows
  • Checking the Status of the Management Agent on Windows
• Controlling the Oracle Management Service
  • Controlling the Management Service on UNIX
    • Using emctl to Start, Stop, and Check the Status of the Oracle Management Service
  • Controlling the Management Service on Windows
• Controlling Fusion Middleware Control
• Controlling the Database Control on UNIX
  • Starting the Database Control on UNIX
  • Stopping the Database Control on UNIX
  • Starting and Stopping the Database Control on Windows
• Guidelines for Starting Multiple Enterprise Manager Components on a Single Host
• Starting and Stopping Oracle Enterprise Manager 11g Grid Control
  • Starting Grid Control and All Its Components
  • Stopping Grid Control and All Its Components
• Additional Management Agent Commands
  • Uploading and Reloading Data to the Management Repository
  • Specifying New Target Monitoring Credentials
    • Using the Grid Control Console to Modify the Monitoring Credentials
    • Using the Enterprise Manager Command Line to Modify the Monitoring Credentials
  • Listing the Targets on a Managed Host
  • Controlling Blackouts
  • Changing the Management Agent Time Zone
  • Reevaluating Metric Collections
• emctl Commands
• Using emctl.log File

8 Information Publisher

• About Information Publisher
• Out-of-Box Report Definitions
• Custom Reports
  • Creating Custom Reports
  • Report Parameters
  • Report Elements
• Scheduling Reports
  • Flexible Schedules
  • Storing and Purging Report Copies
  • E-mailing Reports
• Sharing Reports

9 Sizing Your Enterprise Manager Deployment

• Oracle Enterprise Manager Grid Control Architecture Overview
• Enterprise Manager Grid Control Sizing and Performance Methodology
  • Step 1: Choosing a Starting Platform Grid Control Deployment
    • Network Topology Considerations
  • Step 2: Periodically Evaluate the Vital Signs of Your Site
  • Step 3: Use DBA and Enterprise Manager Tasks To Eliminate Bottlenecks Through Housekeeping
    • Online Weekly Tasks
    • Offline Monthly Tasks
  • Step 4: Eliminate Bottlenecks Through Tuning
    • High CPU Utilization
    • Loader Vital Signs
    • Rollup Vital Signs
    • Rollup Process
    • Job, Notification, and Alert Vital Signs
    • I/O Vital Signs
    • The Oracle Enterprise Manager Performance Page
  • Step 5: Extrapolating Linearly Into the Future for Sizing Requirements
• Oracle Enterprise Manager Backup, Recovery, and Disaster Recovery Considerations
  • Best Practices for Backup
  • Best Practices for Recovery
    • Recovering the Management Repository
    • Recovering the Oracle Management Service
    • Recovering the Oracle Management Agent
    • Preventing Data Loss and Down Time While Switching From a Non-shared File System to a Shared File System
  • Best Practice for Disaster Recovery (DR)
    • Management Repository
    • Oracle Management Service
    • Management Agent

10 Maintaining and Troubleshooting the Management Repository

• Management Repository Deployment Guidelines
• Management Repository Data Retention Policies
  • Management Repository Default Aggregation and Purging Policies
  • Management Repository Default Aggregation and Purging Policies for Other Management Data
  • Modifying the Default Aggregation and Purging Policies
  • Modifying Data Retention Policies When Targets Are Deleted
  • How to Modify the Retention Period of Job History
• Changing the SYSMAN Password
  • Overview of the MGMT_VIEW User
• Dropping and Recreating the Management Repository
  • Dropping the Management Repository
  • Recreating the Management Repository
    • Using the RepManager Script to Create the Management Repository
    • Using a Connect Descriptor to Identify the Management Repository Database
• Troubleshooting Management Repository Creation Errors
  • Package Body Does Not Exist Error While Creating the Management Repository
  • Server Connection Hung Error While Creating the Management Repository
  • General Troubleshooting Techniques for Creating the Management Repository
• Cross Platform Enterprise Manager Repository Migration
  • Common Prerequisites
  • Methodologies
    • Cross Platform Transportable Tablespaces
    • Data Pump
    • Export/Import
  • Post Migration Verification
• Improving the Login Performance of the Console Home Page

11 Locating and Configuring Enterprise Manager Log Files

• Locating and Configuring Management Agent Log and Trace Files
  • About the Management Agent Log and Trace Files
  • Locating the Management Agent Log and Trace Files
  • About Management Agent Rollover Files
  • Controlling the Size and Number of Management Agent Log and Trace Files
  • Controlling the Contents of the Management Agent Trace File
  • Controlling the Size and Number of Fetchlet Log and Trace Files
  • Controlling the Contents of the Fetchlet Trace File
• Locating and Configuring Oracle Management Service Log and Trace Files
  • About the Oracle Management Service Log and Trace Files
  • Locating Oracle Management Service Log and Trace Files
  • Controlling the Size and Number of Oracle Management Service Log and Trace Files
  • Controlling the Contents of the Oracle Management Service Trace File
  • Controlling the Oracle WebLogic Server and Oracle HTTP Server Log Files

12 Monitoring WebLogic Domains

• Updating the Agent Truststore
  • Importing a Demo WebLogic Server Root CA Certificate.
  • Importing a Custom Root CA Certificate
• Changing the Default AgentTrust.jks Password Using Keytool
• Discovering and Monitoring weblogic domains where Admin Channel is enabled
• Collecting JVM Performance Metrics for WebLogic Servers
  • Setting the PlatformMBeanServerUsed Attribute
  • Activating Platform MBeans on WebLogicServer 9.x to 10.3.2 versions

Part II Advanced Topics

13 Managing Compliance

• Compliance Overview
• Compliance Management
  • Accessing Compliance Management Pages in Grid Control
  • Investigating Policy Violations and Policy Group Evaluation Results
  • Assessing Security
  • Viewing Policy Violations Results
  • Policy Violations Reports
• Setting Up Compliance Evaluations
  • Scheduling an Evaluation
  • Viewing Policy Group Evaluation Results
  • Out-of-Box Policies and Policy Groups
  • Customizing Policies
    • Defining Corrective Actions
    • Using Templates for Monitoring
• Policy Groups Provided by Oracle
  • Secure Configuration for Oracle Database
  • Secure Configuration for Oracle Real Application Cluster
  • Secure Configuration for Oracle Listener

14 Configuring Services

• Summary of Service Management Tasks
• Setting up the System
• Creating a Service
• Configuring a Service
  • Availability Definition
  • Performance Metrics
  • Usage Metrics
  • Business Metrics
  • Service Tests and Beacons
    • Configuring the Beacons
    • Configuring Windows Beacons for Web Transaction (Browser) Playback
  • Root Cause Analysis Configuration
    • Getting the Most From Root Cause Analysis
• Recording Web Transactions
• Monitoring Settings
• Configuring Aggregate Services
• Configuring End-User Performance Monitoring
  • Configuring End-User Performance Monitoring Using Oracle HTTP Server Based on Apache 2.0 or Apache HTTP Server 2.0
    • Setting up the Third Party Apache Server
  • Configuring End-User Performance Monitoring Using Oracle Application Server Web Cache
    • Configuring Oracle Application Server Web Cache 10.1.2
    • Configuring Oracle Application Server Web Cache 9.0.4
    • Configuring End-User Performance Monitoring Using Earlier Versions of Oracle Application Server Web Cache
    • Configuring End-User Performance Monitoring Using Standalone Oracle Application Server Web Cache
  • Configuring End-User Performance Monitoring for Web Page Extensions
  • Configuring End-User Performance Monitoring for Web Pages Having the Same URI
  • Starting and Stopping End-User Performance Monitoring
  • Verifying and Troubleshooting End-User Performance Monitoring
  • Enabling End-User Performance Monitoring for Third-Party Application Servers
• Managing Forms Applications
  • Recording and Monitoring Forms Transactions
    • Setting the Permissions of the .java.policy File
    • Using a Trusted Enterprise Manager Certificate
    • Adding a Forms Certificate to the Enterprise Manager Agent
    • Configuring the Forms Server
    • Installing the Transaction Recorder to Record and Play Back Forms Transactions
  • Monitoring the End-User Performance of Forms Applications
    • Configuring the Forms Server for End-User Performance Monitoring
    • Configuring the OracleAS Web Cache
    • Configuring the Oracle HTTP Server / Apache HTTP Server
    • Starting and Stopping End-User Performance Monitoring
• Configuring OC4J for Request Performance Diagnostics
  • Selecting OC4J Targets for Request Performance Diagnostics
  • Configuring Interactive Transaction Tracing
  • Configuring OC4J Tracing for Request Performance Data
  • Additional Configuration for Monitoring UIX Applications
• Setting Up Monitoring Templates
  • Configuring Service Tests and Beacons
• Configuring Service Levels
  • Defining Service Level Rules
  • Viewing Service Level Details
• Configuring a Service Using the Command Line Interface
• Troubleshooting Service Tests
  • Verifying and Troubleshooting Forms Transactions
    • Troubleshooting Forms Transaction Playback
    • Troubleshooting Forms Transaction Recording
    • Troubleshooting End-User Performance of Forms Transactions
  • Verifying and Troubleshooting Web Transactions

15 Extending Enterprise Manager

• Benefits of Extending Enterprise Manager
• More Extensibility Information

Part III Enterprise Manager High Availability

16 High Availability Solutions

• Latest High Availability Information
• Defining High Availability
  • Levels of High Availability
• Determining Your High Availability Needs
• RTO, RPO, and Availability Levels

17 High Availability: Single Resource Configurations

• About Single Resource Configurations
• Deploying Grid Control Components on a Single Host
• Backup and Recovery
  • Repository Backup and Recovery
    • Repository Backup
    • Repository Recovery
    • Recovery Scenarios
  • Oracle Management Service Backup and Recovery
    • Backing Up the OMS
    • Recovering the OMS
    • OMS Recovery Scenarios
  • Agent Backup and Recovery
    • Backing Up Agents
    • Recovering Agents
    • Agent Recovery Scenarios
  • Recovering from a Simultaneous OMS-Repository Failure
    • Collapsed Configuration: Incomplete Repository Recovery, Primary OMS on the Same Host
    • Distributed Configuration: Incomplete Repository Recovery, Primary OMS and additional OMS on Different Hosts, SLB Configured
  • EMCTL High Availability Commands

18 High Availability: Multiple Resource Configurations

• Managing Multiple Hosts and Deploying a Remote Management Repository
• Using Multiple Management Service Installations
• High Availability Configurations
  • Configuring the Management Repository
    • Post Management Service - Install Management Repository Configuration
  • Configuring the Management Services
    • Management Service Install Location
    • Configuring the First Management Service for High Availability
    • Configuring Management Service to Management Repository Communication
    • Configuring Shared File System Loader
    • Configuring Software Library
    • Configuring a Load Balancer
  • Configuring Additional Management Services
    • Installing a Fresh Additional Management Service According MAA Best Practices
    • Retrofitting MAA Best Practices on Existing Additional Management Service
  • Configuring the Management Agent
    • Load Balancing Connections Between the Management Agent and the Management Service
  • Disaster Recovery
    • Prerequisites
    • Setup Standby Database
    • Setup Standby Management Service
    • Switchover
    • Failover
    • Automatic Failover
• Installation Best Practices for Enterprise Manager High Availability
  • Configuring the Management Agent to Automatically Start on Boot and Restart on Failure
  • Configuring Restart for the Management Agent
  • Installing the Management Agent Software on Redundant Storage
  • Install the Management Service Shared File Areas on Redundant Storage
• Configuration With Grid Control
  • Console Warnings, Alerts, and Notifications
  • Configure Additional Error Reporting Mechanisms
  • Component Backup
  • Troubleshooting
    • Upload Delay for Monitoring Data
    • Notification Delay of Target State Change
• Configuring Oracle Enterprise Manager for Active and Passive Environments
• Using Virtual Host Names for Active and Passive High Availability Environments in Enterprise Manager Database Control
  • Set Up the Alias for the Virtual Host Name and Virtual IP Address
  • Set Up Shared Storage
  • Set Up the Environment
  • Ensure That the Oracle USERNAME, ID, and GROUP NAME Are Synchronized on All Cluster Members
  • Ensure That Inventory Files Are on the Shared Storage
  • Start the Installer
    • Windows Specific Configuration Steps
  • Start Services
• Configuring Grid Control Repository in Active/Passive High Availability Environments
  • Installation and Configuration
  • Set Up the Virtual Host Name/Virtual IP Address
  • Set Up the Environment
  • Synchronize Operating System User IDs
  • Set Up Inventory
  • Install the Software
    • Windows Specific Configuration Steps
  • Startup of Services
  • Summary
• How to Configure Grid Control OMS in Active/Passive Environment for High Availability Failover Using Virtual Host Names
  • Overview and Requirements
  • Installation and Configuration
  • Setting Up the Virtual Host Name/Virtual IP Address
  • Setting Up Shared Storage
  • Setting Up the Environment
  • Synchronizing Operating System IDs
  • Setting Up Shared Inventory
  • Installing the Software
    • Windows Specific Configuration Steps
  • Starting Up Services
  • Summary
• Configuring Targets for Failover in Active/Passive Environments
  • Target Relocation in Active/Passive Environments
  • Installation and Configuration
    • Prerequisites
    • Configuration Steps
  • Failover Procedure
  • Fallback Procedure
  • EM CLI Parameter Reference
  • Script Examples
    • Relocation Script
    • Start Listener Script
    • Stop Listener Script

19 Management Agent and Management Services

• Reconfiguring the Oracle Management Agent
  • Configuring the Management Agent to Use a New Management Service
  • Securing the Management Agent
  • Changing the Management Agent Port
  • Controlling the Amount of Disk Space Used by the Management Agent
  • About the Management Agent Watchdog Process
  • Setting the Management Agent Time Zone
    • Understanding How the Management Agent Obtains Time Zone Information
    • Resetting the Time Zone of the Management Agent Due to Inconsistency of Time Zones
    • Troubleshooting Management Agent Time Zone Problems
  • Adding Trust Points to the Management Agent Configuration
• Reconfiguring the Oracle Management Service
  • Configuring the Management Service to Use a New Management Repository
    • Changing the Repository Properties
    • About Changing the Repository Password
  • Configuring the Management Service to Prompt You When Using Execute Commands
  • Troubleshooting Management Service Time Zone Problems

Index