Look at all the pretty lights

This week, I needed to reconfigure a previously-used Cyclades ACS console server. Finding the right adapters was easy enough, but getting the USB serial dongle to play was the tricky bit. Without a signal tracer, I was stuck with the evidence... "it's not working"...and I was trying different hardware combinations, and a few software changes. I wasted 2.5 hours going through different combinations, and I put a reminder in my calendar to bring my signal tracers and my Serial Doctors Bag the next day.

Once I had my trusty signal tracers, it was a matter of 10 minutes to find the proper USB settings, COM port configuration in my terminal emulator, and start the reconfiguration process. Blinky lights will save you time, every time. Good quality signal tracers are worth a high cost, because they will save you time often, and serve you well for many years. One day, blinky lights will probably save my life...

So, having solved Problem #1, and with kit in hand, I tackled Problem #2;

Years ago, a couple of hundred APC smart power strips (OK, "0-RU, metered Power Distribution Units") were installed in a data center. Fast forward to last week, when one unit is failing to respond to SNMP after a high-temperature 'event' in the room. Since it was in critical service, we wanted to try using the serial console to see if the brains were still alive. Unfortunately, nobody could find a serial cable for the job, and APC doesn't want folks making their own cables so APC doesn't post any signaling clues. I did find one website with clues about the wiring for the 940-0144 cable.

The clues indicated that the 6-pin RJ-14 interface was, indeed, RS-232 (and not TTL, for example), I could do some experimenting. A new toy in my Doctors Bag (not in the pictures (yet), is a MODAPT from Siemon. With this, and my trusty Digital Volt Meter (DVM), I was able to confirm some signalling that wasn't mentioned in the LuxNET web page.

Pin1 -2v (in reference to Pin2)
Pin2 GND (0v potential to Pin5)
Pin3 TxD output (-5.5v in reference to Pin2)
Pin4 RxD input (0.2V in reference to Pin2)
Pin5 GND (0v potential to Pin2)
Pin6 -2v (in reference to Pin2)

With this information, I could then make an adapter cable, and then use the RJ45 signal tracer to confirm that the signals are on the right pins. (I found that I could use just 3 wires, Ground - RxD - TxD) for communications to the APC AP7868 rPDU. I did not need to loop RJ-14 pins 1 and 6, and I didn't need to connect RJ-14 pins 1 or 6 to ground to establish communication.)

I'll update my APC console page and my Signals page soon with some of this information.


APC Serial Ports, and websites, wikis, and backups, oh my!

How many of you use APC brand UPS gear? Raise your hands! Ok, you can put your hands down.

Now, how many of you have connected the UPS serial port to your computer, to get power information, or to have the UPS tell your computer to shut down? Not so many? How many tried, but had trouble? 

Question 1: Were you using the APC-supplied cable (with the correct end plugged into the UPS)?

I ask, because the APC serial ports are unusual! If you use one of the cables you have lying around your shop, there is a good chance that the UPS shut down, and would not turn on again until you disconnected the cable. 

When I first tried this, the cable was missing, and I tried various cables in the shop (because I didn't have passive signal tracers at the time). Nothing worked, and I sometimes lost power. 

When I had another UPS, with the APC cable, I found that the cable wasn't long enough, and some of the cables I tried using to extend the run caused troubles. That's when I dug deeper into the problem. Here's what I've found;
  • The APC port puts data and ground on unusual pins!
  • The UPS uses the two outputs and inputs for other purposes!
You can actually make a "Big Red Switch" for your small APC, using a couple wires and the DE-9 connection. Because of the signals being on unusual pins, when you plug in a PC with a non-APC cable, you are actually applying a "shut-down" hard-wired signal to the UPS.

I made an APC Clues web page, documenting the findings, in late 2007. But, for some reason, I guess I didn't post it. Only when I needed the information again (and couldn't find it) did the panic set in, because some of the legacy information is no longer on the web.

That brings us to....

Question 2: How many of you are making backups of your critical files?

You can replace the word "critical" with Photos, Homework, Research, Family Tree, Tax Info, you get the idea. The data that you do not want to lose in a hard disk crash, or on a lost Flash Memory Drive. Where do you keep it, and how often do you update it?

I use an automated system, backing up critical files to a disk "in the cloud", for $99 a year. This runs every time I start up my computer, and I put a lot of faith in it. But, I need to pay money to keep it alive.
I also make manual copies to a 1 GB external hard disk (but I have to remember to move new files). I should really set this up to run automatically.
For some of my hobbies, and console stuff, I put copies of the web. (Technically, I made them for display on the web, so I guess the copies on my home machine are my backups.)
For some data sets (photos for some of my hobbies or family, or my console-related files, for example) will be occasionally copied to a CD or a DVD, and distributed to friends, family, or colleagues. While it isn't complete, this may be one of my best backup methods!

My point here is that disks die, memory sticks get lost, laptops get stolen, and even servers or hosting companies die without prior notice. Data stored on websites may still be transient. If it is your data, you should make multiple, diverse copies, and make more copies over time as new media becomes popular. (And, if the data is sensitive, ENCRYPT IT!)

When I couldn't find my APC web page (I was certain that I had made one), I started checking the Wacky Wayback Machine to try to find the information. Sometimes this comes up empty, if the domain/pages you seek were not deemed interesting enough.

In closing, if you have any APC UPS gear, please check out my APC Clues page. Save a printed copy, because you may want that information again someday.  ;)




How about Ubiquitous USB Serial Consoles

Happy New Year! Sorry I've missed posting in a couple months.

It's odd to me, when sysadmins tell me that "Serial Consoles are Dead. USB killed them." But, when I ask them what they mean, I get different answers.

The first meaning is: "RS-232 is dead, long live USB!" So, then I ask them what they do when they need to get to the serial console, and they tell me about needing to find a USB-to-DE9 serial adapter, get that to work, and point the serial console getty to that device. ("I bet that's tough to do when you are having trouble with a server" I offer. They usually agree that it is...)

The second meaning is: "You don't need DE-9 connectors anymore, you can do it all with USB". This has more insidious meanings...

2a) Most new servers don't have a DE-9 connector on the chassis, just USB ports.

2b) If I need a serial port, I can add a USB-to-DE9 dongle, and use minicom.

Have you ever tried plugging in a USB-connected external modem, and then tried to dial into it, to access the serial console (getty) on your server? The adventure is fraught with problems, which must be solved with a Crash Cart or other form of KVM or remote desktop (which means you need to think about it BEFORE you have a problem, so that it's a useful avenue when trouble presents itself).

Remember, the serial console has been your friend, when the network or the network-facing configuration goes awry. Do you think USB is ready to replace that role? Can you go to Fry's, and buy a USB-to-USB cable which you can simply plug in between two servers, and automatically establish a "null-modem"-type data path? I haven't seen one. Why Not?

Someone needs to develop a little, microprocessor pod, with two USB cables. Power the unit from the 5-volt signal from either USB device. Maybe even add some TX and RX data status LEDs, to show which USB cable is talking. You don't even need to do the conversion from TTL to RS-232 and back...just keep it all at TTL levels! But the microprocessor needs to be able to talk to each USB port when the computer inquires "Who are you, what are you?"...this is the key!

USB relies on drivers, residing on your computer(s), to recognize the ID sent from a USB-attached device when you first plug it in. To make things easy, many basic drivers are bundled into Operating System packages, which lets you plug in many devices without needing to install the drivers first. (This is the difference between Play and Pray in "Plug-n-Play" architecture.)

Looking at the basic USB-to-DE9 dongles, most are derivative works from two basic hardware types. For example, since Brands A, G, M and Q all use "Chip X", and Brands D, I and S all use "Chip Z", it's easy to see that just having two basic (generic) drivers will let you plug in most of the devices from these 7 brands of dongles. Getting Microsoft, and the various Linux distributions is an important step in making things easier for the folks who use these operating systems with serial devices.

But, if you make this USB cable I'm describing, and you try to use either of these USB-to-Serial chips as a base, you will find that most systems will think that you are attaching a plain serial device, and they'll use the generic driver. So, you'll need a bit more engineering...you need a driver that will determine if it needs to find (or create) a getty process, and connect it to the I/O address for the cable. Since you are going to need to make your own driver, you just gave yourself the flexibility to define how this type of cable communicates with the computers.

In a basic "Crash Cart" scenario, I want to plug the "Server" cable into my server chassis. If there is a console getty process running, I want it to be directed to this new device (even if it thinks it was connected to some other previous USB-to-Serial dongle, since that may have failed - causing me to need the crash cart).

When I plug in the "Client" cable to my crash cart, or a laptop, I want them to think that it's just another serial port. (For real Ease Of Use, let's default the serial speed to 9600 8N1.) If it's a Windows box, I'd like the option to choose (in the driver configuration) which COM port this device is assigned to (so it's going to ALWAYS be there...even if it needs to steal the assignment from a previous device). I'd also like to be able to associate this interface with an application, so that the OS will try to launch that application when the cable is plugged in.

I want to walk up to a server, just out of the box, with my laptop, and plug this cable into the server and into the laptop...and have the laptop launch my copy of ProCom, or TerraTerm, or HyperTerm...and have the server configure it's getty to talk to the new port, if the OS is healthy.

If we can loop in a few of the big BIOS makers, the BIOS might notice this device on startup, if BIOS redirection has been turned on, and use the cable (as a preference above the usual COM A assignment).

Of course, until we can get to this point, USB hasn't replaced the venerable RS-232 serial port, at least in my data center.

Good luck with this winter!