PM5560 - reasons for disconnection from Ethernet network

[swil]

Hi,

A method to stop random Ethernet Power meters in an Ethernet meter string from disconnecting from the network is sought.

Between the roof of a building and the basement is 13-off  PM5560 meters, connected together by Ethernet cable. These meters then connect to a PM8000 by Ethernet cable and back to a network switch and (PME) EMS server. Each power meter except the one at the end of the run (on the roof)  is used as an Ethernet switch for the next power meter.

Communication parameters in the PM5560 meters are correct (fixed IP addresses, gateway, subnet mask). Different meters in the Ethernet string, at different times, have stopped communicating with the EMS server, while upstream and downstream meters have maintained their connection to the network. The fix has been to power cycle the meters.

Sometimes PM5560 meters can lose their IP addresses. In this case of non-communicating PM5560 meters, fixed IP addresses are retained. Non-communicating meters can't be pinged by the EMS server until they are power cycled.

There are several questions regarding the explanation provided above:

1. What may be causing the problem explained?

2. Is there a limit to the number of Schneider Ethernet meters that can exist on a Cat6a Ethernet string?

3. How can I stop this from happening?

4. If the IT department was to power cycle the switch periodically (eg. once a day or every few days), would this force all devices back online?

[tcrawford_marco]

I have a client having the same issues with several 5560's. Those meters are all connected to Cisco Nexus switches and it seems to be an ARP issue on the network side. As soon as the meter is power cycles and re-broadcasts to the network they are immediately picked up and will operate for another 1-2 months without issue.

I have yet to find a solution other than power cycling the meters.

[Charles_Murison]

Hello @swil ,

1. What may be causing the problem explained?

Hard to say from just this explination. Could be that tcp connection at one end or the other is getting closes and not reset for example. Likely will need to contact technical support, ideally with some wireshark traces before and after the meter stops communicating. Given that meters downstream are able to talk I would say not a physical connection issue.

2. Is there a limit to the number of Schneider Ethernet meters that can exist on a Cat6a Ethernet string?

There is no hard limit that I am aware of. Something to consider is that at the first meter, all traffic to the string is seen and has to go through the same pipe. Which ethernet is fast, there will come a point where performance will degrade with more devices. I cannot say were that point is because many factors need to be included like type of device, how often you need to connunicate etc,

3. How can I stop this from happening?

Need to first determine what is happening to be able to stop.

4. If the IT department was to power cycle the switch periodically (eg. once a day or every few days), would this force all devices back online?

Depends on the device, I cannot speak for all the device types. The PM8000 meter has a setting TCP keep alive, If this is a non 0 number, if the meter sees no activity after this timeout the meter will close the connection. Now the caviate to this is if the software is able to detect the close, for example if the switch is down when the PM8000 sends the close packet, the software may no see this and think the connection is still open and still try to talk to the devices.

In the software, is it possible to close a connection to the device and try to open the connection again? If so, does this restore communication?

[Charles_Murison]

Could this have to do with authentication on the switches? If an ARP is required to restore communication that may be indicate 802.1X with MAB is in use.

[Ramasamy_N · Accepted solution]

Hello @swil 

Can you please share the following data in mail to analyze this issue?

1.Communication network topology. Highlight the effected PM5560 in the diagram. 

2.What is firmware version of PM5560.

3.Copy and share the system.log file (from fw folder in FTP) from PM5560 meters.

Regards, 

Ramasamy N 

[swil]

Thank you Ramasamy. Will get in touch if/when returning to site.