Error:
NTP daemon slewing option check failed on some nodes
Check failed on nodes:
PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
Clock synchronization check using Network Time Protocol(NTP) failed
PRVF-9652 : Cluster Time Synchronization Services check failed
When :
Error at cluster verification check while installing 11g Grid Infrastructure (11.2.0.3) on Linux
Analysis:
During GI installation cluster verification has failed without showing much details in GUI. So $GRID_HOME/cfgtoollogs/oui/installActions*.log has been mined and found cluster verification check failure errors .
Installation did not impacted due to this error, after closing GUI which has shown partial successful completion of installation cluster verification has been invoked manually as below to verify the root cause
$cluvfy stage -post crsinst -n inpudicrmrdbprtrpt1,inpudicrmrdbprtrpt2
Performing post-checks for cluster services setup
Checking node reachability...
Node reachability check passed from node "inpudicrmrdbprtrpt1"
Checking user equivalence...
User equivalence check passed for user "ora11g"
Checking node connectivity...
Checking hosts config file...
Verification of the hosts config file successful
Check: Node connectivity for interface "bond0"
.
.
.
.
.
Checking CSS daemon...
Oracle Cluster Synchronization Services appear to be online.
Cluster manager integrity check passed
Default user file creation mask check passed
Checking cluster integrity...
Cluster integrity check passed
Checking OCR integrity...
Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations
ASM Running check passed. ASM is running on all specified nodes
Checking OCR config file "/etc/oracle/ocr.loc"...
OCR config file "/etc/oracle/ocr.loc" check successful
Disk group for ocr location "+OCR1" available on all the nodes
NOTE:
.
.
.
ONS node application check passed
Checking Single Client Access Name (SCAN)...
Checking TCP connectivity to SCAN Listeners...
.
.
.
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
NTP daemon slewing option check failed on some nodes
Check failed on nodes:
inpudicrmrdbprtrpt2,inpudicrmrdbprtrpt1
PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
Clock synchronization check using Network Time Protocol(NTP) failed
PRVF-9652 : Cluster Time Synchronization Services check failed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.
Post-check for cluster services setup was unsuccessful on all the nodes.
Clock synchronization check between two nodes using NTP got failed
Clock sync component has been verified using cluvfy comp as below
$cluvfy comp clocksync -n all
Verifying Clock Synchronization across the cluster nodes
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
NTP daemon slewing option check failed on some nodes
Check failed on nodes:
inpudicrmrdbprtrpt2,inpudicrmrdbprtrpt1
PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
Clock synchronization check using Network Time Protocol(NTP) failedPRVF-9652 : Cluster Time Synchronization Services check failed
Verification of Clock Synchronization across the cluster nodes was unsuccessful on all the specified nodes.
ntp service status has been verified
$ service ntpd status
ntpd (pid 5513) is running...
$ ps -ef|grep ntp
ntp 5513 1 0 Mar13 ? 00:00:02 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
ora11g 13846 10941 0 12:35 pts/0 00:00:00 grep ntp
$ grep OPTIONS /etc/sysconfig/ntpd
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
NTP service daemon found to be running in -g mode than -x , as the error itself suggest that it should be running in -x mode. -x mode does not allow system clock to be adjusted backward in order to sync cluster node timings
PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
$ more /etc/sysconfig/ntpd
# Drop root to id 'ntp:ntp' by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
/etc/sysconfig/ntpd file has been modified to reflect -x option in all cluster nodes as below
$ more /etc/sysconfig/ntpd
# Drop root to id 'ntp:ntp' by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -x"
ntp service daemon has been restarted
$ sudo service ntpd stop
Shutting down ntpd: [60G[ [0;32m OK [0;39m]
$ sudo service ntpd start
Starting ntpd: [60G[ [0;32m OK [0;39m]
$ ps -ef| grep -i ntpd
ntp 14886 1 0 12:38 ? 00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -x
ora11g 14889 10941 0 12:38 pts/0 00:00:00 grep -i ntpd
Post ntp configuration file changes cluster verification utility has been re-executed
$cluvfy comp clocksync -n all
Verifying Clock Synchronization across the cluster nodes
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
NTP daemon slewing option check passed
NTP daemon's boot time configuration check for slewing option passed
NTP common Time Server Check started...
Check of common NTP Time Server passed
Clock time offset check from NTP Time Server started...
Clock time offset check passed
Clock synchronization check using Network Time Protocol(NTP) passed
Oracle Cluster Time Synchronization Services check passed
Verification of Clock Synchronization across the cluster nodes was successful.
$cluvfy stage -post crsinst -n inpudicrmrdbprtrpt1,inpudicrmrdbprtrpt2
Performing post-checks for cluster services setup
Checking node reachability...
Node reachability check passed from node "inpudicrmrdbprtrpt1"
Checking user equivalence...
User equivalence check passed for user "ora11g"
.
.
.
.
Check of multicast communication passed.
Time zone consistency check passed
Checking Oracle Cluster Voting Disk configuration...
ASM Running check passed. ASM is running on all specified nodes
Oracle Cluster Voting Disk configuration check passed
Checking Cluster manager integrity...
Checking CSS daemon...
Oracle Cluster Synchronization Services appear to be online.
Cluster manager integrity check passed
.
.
.
Checking node application existence...
Checking existence of VIP node application (required)
VIP node application check passed
Checking existence of NETWORK node application (required)
NETWORK node application check passed
Checking existence of GSD node application (optional)
GSD node application is offline on nodes "inpudicrmrdbprtrpt2,inpudicrmrdbprtrpt1"
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
NTP daemon slewing option check passed
NTP daemon's boot time configuration check for slewing option passed
NTP common Time Server Check started...
Check of common NTP Time Server passed
Clock time offset check from NTP Time Server started...
Clock time offset check passed
Clock synchronization check using Network Time Protocol(NTP) passed
Oracle Cluster Time Synchronization Services check passedChecking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.
Post-check for cluster services setup was successful.
Solution:
Modify /etc/sysconfig/ntpd file on all Cluster nodes
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
to
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -x"
And restart ntp service
service ntpd stop
service ntpd start
NTP daemon slewing option check failed on some nodes
Check failed on nodes:
PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
Clock synchronization check using Network Time Protocol(NTP) failed
PRVF-9652 : Cluster Time Synchronization Services check failed
When :
Error at cluster verification check while installing 11g Grid Infrastructure (11.2.0.3) on Linux
Analysis:
During GI installation cluster verification has failed without showing much details in GUI. So $GRID_HOME/cfgtoollogs/oui/installActions*.log has been mined and found cluster verification check failure errors .
Installation did not impacted due to this error, after closing GUI which has shown partial successful completion of installation cluster verification has been invoked manually as below to verify the root cause
$cluvfy stage -post crsinst -n inpudicrmrdbprtrpt1,inpudicrmrdbprtrpt2
Performing post-checks for cluster services setup
Checking node reachability...
Node reachability check passed from node "inpudicrmrdbprtrpt1"
Checking user equivalence...
User equivalence check passed for user "ora11g"
Checking node connectivity...
Checking hosts config file...
Verification of the hosts config file successful
Check: Node connectivity for interface "bond0"
.
.
.
.
.
Checking CSS daemon...
Oracle Cluster Synchronization Services appear to be online.
Cluster manager integrity check passed
Default user file creation mask check passed
Checking cluster integrity...
Cluster integrity check passed
Checking OCR integrity...
Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations
ASM Running check passed. ASM is running on all specified nodes
Checking OCR config file "/etc/oracle/ocr.loc"...
OCR config file "/etc/oracle/ocr.loc" check successful
Disk group for ocr location "+OCR1" available on all the nodes
NOTE:
.
.
.
ONS node application check passed
Checking Single Client Access Name (SCAN)...
Checking TCP connectivity to SCAN Listeners...
.
.
.
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
NTP daemon slewing option check failed on some nodes
Check failed on nodes:
inpudicrmrdbprtrpt2,inpudicrmrdbprtrpt1
PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
Clock synchronization check using Network Time Protocol(NTP) failed
PRVF-9652 : Cluster Time Synchronization Services check failed
Checking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.
Post-check for cluster services setup was unsuccessful on all the nodes.
Clock synchronization check between two nodes using NTP got failed
Clock sync component has been verified using cluvfy comp as below
$cluvfy comp clocksync -n all
Verifying Clock Synchronization across the cluster nodes
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
NTP daemon slewing option check failed on some nodes
Check failed on nodes:
inpudicrmrdbprtrpt2,inpudicrmrdbprtrpt1
PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
Clock synchronization check using Network Time Protocol(NTP) failedPRVF-9652 : Cluster Time Synchronization Services check failed
Verification of Clock Synchronization across the cluster nodes was unsuccessful on all the specified nodes.
ntp service status has been verified
$ service ntpd status
ntpd (pid 5513) is running...
$ ps -ef|grep ntp
ntp 5513 1 0 Mar13 ? 00:00:02 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
ora11g 13846 10941 0 12:35 pts/0 00:00:00 grep ntp
$ grep OPTIONS /etc/sysconfig/ntpd
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
NTP service daemon found to be running in -g mode than -x , as the error itself suggest that it should be running in -x mode. -x mode does not allow system clock to be adjusted backward in order to sync cluster node timings
PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
$ more /etc/sysconfig/ntpd
# Drop root to id 'ntp:ntp' by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
/etc/sysconfig/ntpd file has been modified to reflect -x option in all cluster nodes as below
$ more /etc/sysconfig/ntpd
# Drop root to id 'ntp:ntp' by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -x"
ntp service daemon has been restarted
$ sudo service ntpd stop
Shutting down ntpd: [60G[ [0;32m OK [0;39m]
$ sudo service ntpd start
Starting ntpd: [60G[ [0;32m OK [0;39m]
$ ps -ef| grep -i ntpd
ntp 14886 1 0 12:38 ? 00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -x
ora11g 14889 10941 0 12:38 pts/0 00:00:00 grep -i ntpd
Post ntp configuration file changes cluster verification utility has been re-executed
$cluvfy comp clocksync -n all
Verifying Clock Synchronization across the cluster nodes
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
NTP daemon slewing option check passed
NTP daemon's boot time configuration check for slewing option passed
NTP common Time Server Check started...
Check of common NTP Time Server passed
Clock time offset check from NTP Time Server started...
Clock time offset check passed
Clock synchronization check using Network Time Protocol(NTP) passed
Oracle Cluster Time Synchronization Services check passed
Verification of Clock Synchronization across the cluster nodes was successful.
$cluvfy stage -post crsinst -n inpudicrmrdbprtrpt1,inpudicrmrdbprtrpt2
Performing post-checks for cluster services setup
Checking node reachability...
Node reachability check passed from node "inpudicrmrdbprtrpt1"
Checking user equivalence...
User equivalence check passed for user "ora11g"
.
.
.
.
Check of multicast communication passed.
Time zone consistency check passed
Checking Oracle Cluster Voting Disk configuration...
ASM Running check passed. ASM is running on all specified nodes
Oracle Cluster Voting Disk configuration check passed
Checking Cluster manager integrity...
Checking CSS daemon...
Oracle Cluster Synchronization Services appear to be online.
Cluster manager integrity check passed
.
.
.
Checking node application existence...
Checking existence of VIP node application (required)
VIP node application check passed
Checking existence of NETWORK node application (required)
NETWORK node application check passed
Checking existence of GSD node application (optional)
GSD node application is offline on nodes "inpudicrmrdbprtrpt2,inpudicrmrdbprtrpt1"
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check passed for "ntpd"
Check for NTP daemon or service alive passed on all nodes
NTP daemon slewing option check passed
NTP daemon's boot time configuration check for slewing option passed
NTP common Time Server Check started...
Check of common NTP Time Server passed
Clock time offset check from NTP Time Server started...
Clock time offset check passed
Clock synchronization check using Network Time Protocol(NTP) passed
Oracle Cluster Time Synchronization Services check passedChecking VIP configuration.
Checking VIP Subnet configuration.
Check for VIP Subnet configuration passed.
Checking VIP reachability
Check for VIP reachability passed.
Post-check for cluster services setup was successful.
Solution:
Modify /etc/sysconfig/ntpd file on all Cluster nodes
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
to
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -x"
And restart ntp service
service ntpd stop
service ntpd start
No comments:
Post a Comment