Troubleshooting Basics - Tips to Troubleshoot Telecom & IT Problems

Think of this page as learning "Service Call Stress Reduction"!

What's the first thing NOT to do?

Don't replace the KSU!

Read the FREE information we have here, think about the problem, and do some testing...

With this information, your stress level will go way down because you'll know how to find what's causing your customer's problem.

We don't have "magic" gizmos. If you look at our web pages, read our Knowledgebase, have some spares, and have a fairly good digital meter, you can probably fix a lot of the problems you encounter - for FREE!

If you want to be a successful Repairman, you should remember and practice ONE THING...

START AT THE BEGINNING

We're not talking about saving a few minutes by doing basic troubleshooting instead of guessing. We're talking about saving many hours of wasted time, when you could have found the problem quickly by just starting at the beginning (I speak from experience!)

In order to start at the beginning, you need to have a clear picture in your mind of how the gizmo you're fixing is connected, so you know where the beginning is, and the next thing to check.

To fix a phone line, it might be the telephone company or channel bank to the NI (Network Interface), to the lightning protector and ground, to a mod jack, to the CO line card, to the CO line (or system wide) programming on the system for things like attenuation, impedance, echo cancellation, and CPC, etc.

You also need to think about any external stuff connected to the phone system, like the AC power, battery backup, ground, other phone lines, stations, paging etc. All the external gizmos connected to a phone system can be causing a problem on the phone system - so you want to get them disconnected to make absolutely sure they're not causing your problem.

If you leave out something, Murphy's Law says that will be the problem (that you'll find many hours later). Making yourself a list or drawing a picture of how everything is connected might be a good idea when you walk into the job.

Simply stripping a system down and adding everything back one thing at a time until the problem returns, is the best way to troubleshoot a problem.

I know you don't want to do this. You'd like to poke around at the easiest stuff first. You can do that, but if you find the problem it will usually take longer than if you started at the beginning and did your troubleshooting in a methodical manner.

Fixing a system today by stripping it down and starting at the beginning is much easier than in the old days of 1A2 (with 25 to 100 pair cable). On a 1A2 system, you could have 30 or more phones that were all the lines were bridged together. If a line wasn't working right, you could disconnect the line card in the 1A2 system from all the phones by pulling one set of jumpers. If the problem went away, one (or more?) of those 30 phones had a problem (often a water damaged 25 pair connector that got wet when mopped).

You could guess at which phone it was and start taking apart connectors to see if they were green inside (you couldn't see the water damage from the outside), and if you got it on the first five tries you would have gone out and played the lottery. It was your lucky day.

The only reasonable way to find the problem was to start removing the 3 pair jumpers to a line on a phone, one at a time (marking them so you could put them back in the same position on the 66B block). Remove a jumper, test to see if the problem cleared, and if it didn't go on to the next jumper. If it cleared, you trace that jumper to the cable, then look at the floor plan to see where the cable was. If you were having a really unlucky day, the bad phone was the 30th jumper.

Modern systems are much easier!

So, you've got a strange problem. Since you've probably got dozens or hundreds of the particular model of system you're working on installed, and the problem isn't occurring at those other places, there must be something different about the place you're at. All you have to do is find what's different!

Phone system hardware (from traditional phone system manufacturers) is more dependable than ever. The cards and power supplies just don't break like they did in the 80's and 90's. While the problem you're looking at might be caused by a bad card or power supply, it may just be an interaction with something that's connected to that particular phone system, at that particular site.

Since the phone system by nature has to connect to the outside world, the interface to the outside world is likely to be causing some problems. It's easy to check the power on a system by simply running the system on the battery backup to see if the problem clears (it's not a bad idea to carry a cheap battery backup in your truck for testing). Removing the system ground is also a good thing to do. The AC ground is already disconnected as soon as you pull the plug from the wall on the battery backup/UPS, to run the system on batteries. Unless you put it in yourself, it's hard to know whether a ground is good! Just because it's a green wire, it doesn't mean it's actually connected to a good ground, or even anything at the other end.

Fixing an odd problem could be as simple as reversing the tip and ring on a CO line (especially from a channel bank or VoIP box), an external page port, external bells, or an MOH/BGM port. Ground loops can cause strange problems. Leaving one external device connected to a system that you thought you had stripped down could cost you hours.

Although it might seem logical to assume something in your troubleshooting process (like "I've never seen one of those go bad" or even "The AC outlet must be wired right"), assuming anything without checking it in its turn in the troubleshooting process has led a lot of guys to chase their tails for hours or days.

We can all remember chasing our tails! Just about every time I've assumed something, I got sidetracked and the problem took longer to fix. Sure, I still do it sometimes (it's human nature), but not very often.

While it might sound harder to always Start at the Beginning, you get used to troubleshooting that way and wonder why you wasted all that time in the past!

One other thing... Always try your new tester at your office or home before trying to use it at the customer's site!

Once you know what your meter readings are supposed to look like, you'll feel confident using testers to solve your customer's problems.

If you remember, believe, and practice what you've just read... you'll be able to fix anything!

I've walked into a lot of jobs where the system was down, and the customer was going nuts. If I took the cue and started acting nervous and stressed, things would have gotten worse. When I go on a service call, the only thing I know for sure is that the problem can be fixed. There's no question about it. That pretty much gets rid of the stress for me, even if the customer is acting nuts.

On the drive over to the service call, I get a picture of the system in my mind and think about where to start and how to proceed. I do get screwed up occasionally when the person who took the service call wrote down the wrong problem or didn't ask enough questions. Sometimes I'll call the customer when I'm on the way just to verify the problem (and let them know I'll be there shortly). Sometimes I'll go pick up some parts before I go on the call. The amount of traffic in bigger cities makes it pretty tough to go back to the shop for parts, as it would in a more rural area where it might be a 100 or more mile drive.

I personally would never agree to service a system that I didn't have spares for. If you need the spares from the trade-in on a system you're proposing, that's a good reason to give someone a good price on the trade-in. It's still going to be pennies on the dollar. Considering how many used systems are out there from brokers (or eBay), having a whole working system (with at least one station and trunk card) as a spare isn't expensive. It turns a stressful job (telephone repair) into a fun job. You don't need to have every software revision, just something to do troubleshooting and substitution with, and something to get the customer up and running until you get the right software version.

Most Interconnects can't have full systems as truck stock for every type of system in your base. If you have a few trucks, it's not a bad idea to keep one type on each truck, so they can bring it over where needed. Making a deal with a cab company or delivery service to deliver a KSU, common cards, and power supply to a job site in an emergency is also a good idea (as it is when you forget something on a cutover).

Every Interconnect has to deal with the problem of bad cards getting mixed up with good ones. In the heat of battle, while you're swapping cards, it's not hard to lose track of them since they all look the same. I carry both gray electrical tape, and tags with string (don't use tags with wire!). The gray tape is easy to stick on a card or the bottom of a phone, so I can write down what I did and the date and time I did it when I swap something. If there isn't already a tag on a card I take out of truck stock, I put some tape or a tag on it so I know it was from the truck.

If you do lose track of which card is which, having the whole system lets you test the cards in your shop before returning them to stock. There's nothing that screws up a repairman more than having bad cards for spares. It's something to avoid at all costs.

I get a lot of calls from guys asking me all the possible things that could cause a problem before they even go on a service call. Then they want me to give them the solutions to each possible problem. That would be really hard! You're causing yourself an incredible headache to do that before a service call. You've got to start at the beginning, find the cause of the symptom, and then start trying to figure out what's causing it.

Every service call I go on is a challenge, but it's fun because I know it'll be fixed soon, and I get to go on to the next job. It's kind of like being a detective and surgeon in one job - but a lot safer! What scares me is seeing a doctor that doesn't know how to troubleshoot... And there are a lot of them out there.

Troubleshooting Telephone Lines

Disconnect the phone line from the equipment and test the line with your Butt-set. That's basic troubleshooting, but there's more reasons to do it than just to see if it's the Phone Company's problem.

A Butt-set (or old 2500 set) is line powered, so it has no reference to ground or AC. That's important, since some of the problems you encounter will be an interaction between the phone line and the telephone equipment, caused by the telephone equipment's reference to ground and "foreign" voltages (like 110VAC). Every phone system will be slightly imbalanced. That means either the tip or the ring is longer than the other side of the pair, or it can have a small foreign voltage on it from being connected to the phone system. In most cases, this doesn't cause a problem. In some, it does - and those are the cases you'll be on.

Calling the Phone Company and asking them to "Check the Line" or take readings for you is generally a waste of time. They're trained to say, "It tests OK" no matter what. If there's a problem with a Phone Company (CLEC or ILEC) line, it's generally your responsibility to tell the Phone Company exactly why the line is bad. If you give them the numbers, they generally will be more helpful. But hey... if everybody could do this, they wouldn't need us!

A common trick from Phone Company repairmen is to tell you that your phone system is imbalanced or putting out a foreign voltage or ground. No kidding! The testers they use are made for testing lines that aren't terminated. When they hook up their meter (that's made for testing unterminated lines), they're reading the phone system as well as the copper wire. They're supposed to open the line going into the premise before doing most tests so they don't read the CPE (phone system) - which will always read something strange compared to a regular 2500 set (that has no reference to ground or foreign voltage).

I've seen Phone Company guys tell the customer they're disconnecting the line going into the building because it's damaging the telephone network. What a bunch of bull. Some of their test equipment will "stress test" the line, which puts out a high voltage/current to try to cure problems with a copper cable. Although that usually won't hurt a phone or system, they're supposed to disconnect the CPE before they do the test. As you can guess, most Phone Company repairmen don't disconnect the CPE. They don't care. They don't have to.

Fixing phone line problems is usually as simple as finding what's different about that line that's having the problem, compared to others working in that system or at another premise. It's as simple as taking some readings (testing) and comparing the results to other lines at that premise, or at your other customers' sites.

Don't replace the KSU first!

This is what I do first on a strange case of trouble. I've seen a lot of strange stuff, but there's no way I'm smart enough to fix a strange problem without taking some measurements to lead me in the right direction. Once I find the solution, I might try that on the next case of trouble that's exactly the same - but you can really chase your tail for hours if you don't take a couple of minutes to take some basic readings.

This is the easy part:

Click to view and print our Telephone Line Diagnostic Table PDF with Instructions

It's as easy as taking the readings for Loop Current, on and off-hook DC voltage, and Induced AC, and writing the results down on a piece of paper. This lets you compare the results to other lines at the same premise, and at other (working) sites. The problem will probably just jump out at you, and you'll know what to look at to fix the problem.

If you aren't familiar with how to take these readings, see our How to Take Telephone Line Diagnostic Readings article. We also have a 14 minute training video showing how to take these readings.

Filling out this chart is very important these days because customers are switching from the LEC (or ILEC - Incumbent Local Exchange Carrier, or the real Phone Company) to a CLEC (Competitive Local Exchange Carrier, like Mike's Phone Company). In most cases, the customer doesn't call you up to ask if it will work OK. They make the change to save a few bucks and call you when things don't work right (and then spend the next two years' savings on your T&M).

By having a chart of the readings for the old LEC, you'll have a basis of comparison when the customer switches the lines to a CLEC and they don't work right.

You should also check the programming or jumper options for the trunks or trunk cards. You'd want to check the Open Loop Disconnect (CPC) time, any impedance settings (like 600 or 900 ohms), attenuation settings, echo canceller settings, and anything else that can affect the operation of a trunk.

In my opinion, you should automatically fill out the Telephone Line Diagnostic Table for every new installation, whether you're having problems or not. I actually recommend taking the time to do it before you propose a system, so you know if you're in for trouble before you quote the new system! It won't take long before it will take you just a few minutes to fill out the chart, and potential (expensive) problems will jump out at you.

I also recommend having an Installation Checklist before every job you propose. Customers can have expectations about their phone system that they forget to tell you, or your system may just not do something the customer assumes it can do. Using an Installation Checklist can really prevent arguments, especially the ones where the customer says he won't pay you!

We have a bunch of phone line testers that are small, cheap and line powered. We're pretty sure most of you aren't going to go out and buy a $500 tester, but by buying one of our inexpensive testers every once in a while to add to your kit, you buy a $500 tester one function at a time - as you need it.

Most strange phone line problems can be diagnosed by taking these simple readings!

P.S. Sometimes we just get lost in the heat of the battle, and just need to talk about it. This ain't like the old days where you had a boss or tech support guy working for your company you could talk the problem over with.

If you've stripped the system down to the basics and taken all the voltage and current readings and you still can't figure it out, give us a call at 630-980-7710. We're going to ask you for the numbers (results) from the Telephone Line Diagnostic Table to see if we have any ideas how to fix your problem. Getting the numbers and stripping the system down (to start at the beginning) is what I would do if I was there. I'm not a genius, and there's no magic. There's no way I could fix your problem without doing the basics even if I was on the service call myself.

If I go on a service call I know I can fix the problem. Since I know every problem can be fixed, I handle even the worst service calls quickly and without a lot of stress. If you're not excited and stressed (or don't show it), even with the customer breathing down your neck, the customer will have less stress and probably let you do your work. If you explain to the customer what you're doing (show him your chart?), and why you're going about it in the manner you are, that might help calm an excited customer down. Then again, some customers are just going to figure that every phone man knows magic and should be able to fix anything in a minute.

Every service call you get under your belt is one step closer to having a stress-free job!