Unknown Cable Splice Causes Issues

This illustrates the cable splice

A prison facility was having intermittent issues with the locking and unlocking of cells and passageways in one of its’ inmate housing facilities. This is not a good thing to have occurred in a facility such as this! After extensive troubleshooting by the on-site maintenance staff, they decided to call Delta Automation for assistance.  After a quick review of the issues and listening to the steps that the maintenance crew had taken, it was suspected that there were remote I/O issues on this system. Once the remote I/O error statistics in the PLC had been checked, it was confirmed that this was an issue. A date was set to inspect, troubleshoot and certify this remote I/O network. This particular network consisted of a Modicon Quantum hot stand-by processor utilizing dual remote I/O cables. This design is a high up-time and reliable system layout. For a dual system such as this to have intermittent remote I/O issues is highly unusual. For this to occur there would have to be failures on both independent cables simultaneously.  For both cables to be intermittent simultaneously was even more remote.
The testing began at the processor, or head-end of the system. Immediately it was discovered that the in-line hot stand-by terminators were installed backwards. They had been this way since the system was commissioned, several years ago. These terminators were replaced and installed properly. Next it was discovered that a mixture of RG-6 and RG-11 had been utilized for the trunk cable. This is not allowed. The sections of RG-6 trunk were replaced with RG-11 quad shield cable. Once these issues were resolved, the system was tested utilizing a time domain reflectometer (TDR). This piece of equipment allows the user to visually examine the structure of the cable, the connections and the devices (taps and splitters), for any discontinuities (opens/shorts etc.) One part of the certification process that Delta employs is to record the actual distance to each of the taps to assist in locating them in the future. Additionally, an added benefit is to confirm the routing and occurrence/sequence of the drops. In the case of this system, the original drawings indicated the routing to be a certain way when in actuality it was different. This could have been a huge issue should a problem have occurred where the cable routing was involved in troubleshooting the system. It was during this test that a discontinuity was discovered apparently mid-way between two buildings, where no devices should have been located. The issue was noted and would have to be investigated further to determine exactly what it was. Once the primary or A cable was tested to this point, the secondary or B cable was checked. As suspected all of the same installation issues that had existed on cable A were found to be on cable B as well. These also were corrected. Again during the TDR test, a discontinuity was detected at a similar distance on the trunk cable between buildings. This re-confirmed what had been found on the A cable. The cables were visually inspected with great care at every location where they either entered or exited a building. All points were found to be OK. Once this had been completed, the piping and conduits were walked looking for any dents, cuts or physical damage. Some of these cables were underground cables. Obviously these could not be visually inspected. This was of concern in our attempt to locate the discontinuity on each of the cables.  A piping diagram was located on-site and it showed a completely different routing than what had previously been thought. A section of both trunk cables had been routed back into the main control room via a cable tray. It was suspected that, due to the distances observed on the TDR, that most likely due to the installer utilizing a 500-foot roll of cable instead of a 1000-foot roll, needed to splice the trunk cable to get to the next location.  The cable was visually inspected along the cable tray and sure enough, there was an in-line splice on each of the two cables. An undocumented splice. As these splices were handled during their discovery they both showed signs of failure. Subsequent views using the TDR showed that these undocumented splices were intermittent. Most likely this was the major cause for the systems intermittent failures. The splices were cut out, repaired to specification, and then properly documented. The remainder or the testing was completed and the systems were brought back on-line. No further issues have occurred on this system.
This type of installation mistake/change could have continued to cause issues for some time without ever being located.  A network certification utilizing all of the proper test equipment was certainly instrumental in locating and repairing this system.

Advertisements

About deltaautomation

President of Delta Automation, Inc. Working in industrial automation in excess of 30 years
This entry was posted in On-site stories, Service stories and tagged , , , , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s