I have two CS (node 1)Crystal hardware& common Hardware, one has appliance server and configured with few PCS and another CS(node3) has one shelf common hardware. Both nodes are connected with abcf link.
Issue: on 21 Oct , All extensions and trunk went down. Network team has investigated and found no flap in network.
Current status of PCS: Undef: not able to telnet it. Only accepting user name and pass and stuck. I am guessing probably HDD of PCS(shelf 7) become faulty.
What could be possible reason of reboot???? Please let me know if other info is required to investigate further.
Node1 log: Incvisu>
21/10/14 09:32:29 000001M|019/00/0/002|=2:2868=PH:IP Hybrid Link to 3 is broken (type 4 access 1 sig Main)
21/10/14 09:32:29 000001M|019/00/0/002|=3:2867=PH:Hybrid Access (3 1 Main) goes down
21/10/14 09:32:29 000001M|1-3 |=2:2826=PH:Local link [1 3] is down
21/10/14 09:32:29 000001M|---/--/-/---|=3:2832=PH:Node 3 is unreachable
21/10/14 09:32:29 000001M|019/00/0/002|=3:2867=PH:Hybrid Access (3 1 Main) goes down
21/10/14 09:32:29 000001M|019/00/0/002|=3:2865=PH:PLL 0 is down on Hybrid Access (3 1 Main)
21/10/14 09:32:51 000001M|007/--/-/---|=2:2043=Loss of the 7 CRYSTAL
21/10/14 09:32:51 000001M|007/00/-/---|=2:2042=Loss of a GD/GD3 type cpl
21/10/14 09:32:51 000001M|007/01/-/---|=2:2042=Loss of a GA/GA3 type cpl
21/10/14 09:32:51 000001M|007/03/-/---|=2:2042=Loss of a GACONF type cpl
21/10/14 09:32:51 000001M|007/04/-/---|=2:2042=Loss of a PRA type cpl
21/10/14 09:32:51 000001M|007/06/-/---|=2:2042=Loss of a PRA type cpl
21/10/14 09:32:51 000001M|007/06/0/000|=4:2085=Access T2, board (7,6) : synchronization stopped
21/10/14 09:32:51 000001M|007/06/0/000|=4:2080=T2 access is synchronizing on Board (7,7)
21/10/14 09:32:51 000001M|---/--/-/---|=2:2140=Alarm : TRUNK resources quantity critical
21/10/14 09:32:51 000001M|007/07/-/---|=2:2042=Loss of a PRA type cpl
21/10/14 09:32:51 000001M|007/07/0/000|=4:2085=Access T2, board (7,7) : synchronization stopped
21/10/14 09:32:51 000001M|007/04/0/000|=3:2825=PH:Access (7 4 0) goes down
21/10/14 09:32:51 000001M|---/--/-/---|=0:1600=IO1 driver error, 255 No C1 wrong in output
21/10/14 09:32:57 000001M|---/--/-/---|=0:0412=Lost set on IP domain : Domain number 0 pourcent of the set lost 16
21/10/14 09:33:01 000001M|019/00/0/002|=5:2862=PH:Hybrid Access (3 1 Main) is up
21/10/14 09:33:01 000001M|019/00/0/002|=3:2866=PH:Estab Req for PLL 0 on Hybrid Access (3 1 Main)
21/10/14 09:33:01 000001M|019/00/0/002|=5:2864=PH:PLL 0 is up on Hybrid Access (3 1 Main)
21/10/14 09:33:16 000001M|1-3 |=5:2827=PH:Local link [1 3] is up
21/10/14 09:33:16 000001M|---/--/-/---|=5:2846=PH:Node 3 is reachable
21/10/14 09:36:12 000001M|007/00/-/---|=5:0409=The inter-ACT link over IP from (19 1) is up
21/10/14 09:36:12 000001M|007/00/-/---|=0:5857=GD/GA/INTIP/RGD: reason of reboot 2
21/10/14 09:36:27 000001M|011/01/0/105|=4:0389=Sig. on IP set: set demand a reset 00:00:00:00:3b:a7,144330
21/10/14 09:36:29 000001M|007/27/-/---|=4:0260=Beginning of downloading 1652_rva1652_
21/10/14 09:36:29 000001M|007/27/-/---|=5:0261=End of downloading 1652_rva1652_
21/10/14 09:36:30 000001M|007/27/-/---|=4:0260=Beginning of downloading 1653_rva1653_
21/10/14 09:36:30 000001M|007/01/-/---|=3:0747=Config error or Loss of IP connection: 4,(7,1),10.33.114.49,00:80:9f:5c:fd:70
21/10/14 09:36:30 000001M|007/27/-/---|=5:0261=End of downloading 1653_rva1653_
21/10/14 09:36:32 000001M|011/00/0/080|=5:2053=Terminal 80 in service
21/10/14 09:36:41 000001M|007/01/-/---|=4:0740=Beginning of an INT/IP downloading @:00.80.9f.5c.fd.70 (binmg)
21/10/14 09:36:41 000001M|007/01/-/---|=5:0741=End of downloading of an INT/IP board @:00.80.9f.5c.fd.70 (binmg)
21/10/14 09:36:42 000001M|007/00/-/---|=5:2019=GD/GD3 coupler commissioning
Node3 log:
21/10/14 08:32:30 000003M|19/00/0/002|=2:2868=PH:IP Hybrid Link to 1 is broken (type 4 access 1 sig Main)
21/10/14 08:32:30 000003M|19/00/0/002|=3:2867=PH:Hybrid Access (1 1 Main) goes down
21/10/14 08:32:30 000003M|1-3 |=2:2826=PH:Local link [3 1] is down
21/10/14 08:32:30 000003M|--/--/-/---|=3:2832=PH:Node 1 is unreachable
21/10/14 08:32:30 000003M|19/00/0/002|=3:2867=PH:Hybrid Access (1 1 Main) goes down
21/10/14 08:32:30 000003M|19/00/0/002|=3:2865=PH:PLL 0 is down on Hybrid Access (1 1 Main)
21/10/14 08:33:01 000003M|19/00/0/002|=5:2862=PH:Hybrid Access (1 1 Main) is up
21/10/14 08:33:01 000003M|19/00/0/002|=5:2864=PH:PLL 0 is up on Hybrid Access (1 1 Main)
21/10/14 08:33:16 000003M|1-3 |=5:2827=PH:Local link [3 1] is up
21/10/14 08:33:16 000003M|--/--/-/---|=5:2846=PH:Node 1 is reachable
21/10/14 08:33:27 000003M|01/07/0/000|=2:0310=T2 Dlap Released
21/10/14 08:33:27 000003M|01/07/0/000|=4:2113=T2 lapD not established still trying
21/10/14 08:33:27 000003M|01/07/0/000|=3:2825=PH:Access (1 7 0) goes down
21/10/14 08:33:27 000003M|01/07/0/000|=4:2112=NOS alarm on T2 access
21/10/14 08:33:38 000003M|01/07/0/000|=2:2101=T2 access still in state of alarm
21/10/14 08:38:06 000003M|01/07/0/000|=5:2102=T2 access back to normal
21/10/14 08:38:06 000003M|01/07/0/000|=4:2113=T2 lapD not established still trying
21/10/14 08:38:06 000003M|01/07/0/000|=5:0311=T2 Dlap Established
21/10/14 08:38:06 000003M|01/07/0/000|=5:2836=PH:Access (1 7 0) is up
thanks in advance.
