ConsoleTeam

My first impressions of the Redpark serial cable and Get Console

2011-11-02T21:28:00.001-07:00

I'm reviewing them together, as I don't see an easy way to use the cable with another app (yet).
My first attempts were hampered by some Bluetooth weirdness between my iPad 1 and my ZAGGmate keyboard. A hard-reset of the iPad seemed to straighten that out. Since then, I have had good luck using the ZAGGmate with Get Console in both portrait and landscape modes. I'm a bit let down that the top speed is 'only' 57.6 Kbps, as I have a handful of console ports running at 115.2 Kbps. That said, it does perform well at those speeds which it does support.
There is no built-in method to email your log files, but you can copy them to the cut-and-paste buffer,and then paste it into the email app of your choice.
It makes the Escape key sequences visible! This is a BIG plus in my book! If you are working with conserver, this will help you spot devices which are prone to fill your logs with extra characters.
The earlier assertion that the application will only work with Cisco devices seems to be incorrect! That is, with a proper adapter, you can connect the cable to almost any device. Using a CFDTE92 adapter, I can connect to the DE9M com port on most Intel-based servers. (Check my Cisco console connection guide page for more clues, http://www.conserver.com/consoles/Cisco/ciscocons.html )
The cable is about 6' long, with a slender cable. The jacket feels a bit like Teflon, and the RJ45M connector has a molded strain relief, which will likely improve the service life. (That would be great, given the cost of the cable!) It is lightweight, and it looks like a high-quality part. The folks at Redpark have been known for the high quality of their products in the past, and I expect good things from this new product.
I have tried the combination with a variety of non-network devices as well, and I seem to be able to talk to them without needing to do any fussing with the signaling wires. I can't promise that it will work with anything, but the Get Console app running 'stand-alone' (without needing to connect to their website) seems to be a useful basic terminal emulator, if you need a wired serial connection. The cable has made a good addition to my iPad gear bag.
Testing the Get Console website service will need to wait for another post.

October Double-header! Two new useful apps!

2011-10-13T21:47:00.001-07:00

I finally got a Redpark serial cable, and the Get Console app, and it's almost everything I wanted to make my iPad a quick tool for accessing all sorts of serial consoles!

But, I also finally found the Blogger app, which will make it easier to update my blog when I've got stuff to say. AND, I'll be able to do it from my older, pre-3g iPhone! Thank you Google!

The Blogger app also pointed out that I had yet to actually publish a couple older blog articles! That was easy to fix! Now I can get back to testing the Get Console app and the Redpark cable.

Finally, a serial cable for my ipad!

2011-08-10T09:37:00.000-07:00

More precisely, there is a hardware cable (cost is about $60 US) from Redpark, which plugs into the 30-pin ipod/iphone/ipad docking connector, and does all the right stuff to provide an RS-232 serial connection. But, to make practical use of it, you also need the GetConsole iPad application. And it all works without jailbreaking your device!

The application should work on the iPhone as well, but I don't have a cable yet to play with the application. I expect to get them both next month, but I have been tracking the progress getting the cable to the market for a few months!

The Redpark folks have a long history of development for Apple systems, and their legacy includes the Keyspan RS-232 serial adapters, another favorite of mine from many years ago.

The GetConsole application is part of a larger project, though, which may become a great field service tool! I'm a big fan of secure remote access to serial consoles (especially the Conserver console server management application). While the GetConsole application will let you attach your IOS device to the serial console of some other device for local interaction, the application also has the ability to be accessed via the GetConsole website. In this way, a field engineer could use his phone to connect to a serial console, but a senior engineer could then use that the phone network to reach through the phone to interact with the serial port.

I look forward to the chance to play with the cable and the application. I'll likely get the Cisco RJ45 version, since I have a large collection of adapters for them (or you can build your own with these adapter schematics). I'm curious if the application will work on my V2 iPhone (which is limited to running IOS 3.x), or if the application requires IOS 4.x. Either way, I remain excited, and I'll post an update in a month or so.

Ring out the old? Serial Consoles STILL going strong!

2010-12-31T17:47:00.000-08:00

The last couple of weeks have been very busy, helping acquire a small company for a customer. The majority of my role included assessing, and then upgrading the network gear, as well as helping with the various server gear.

After the initial assessment, I installed Conserver on a local host, and I re-deployed a Cisco 3640 with a pair of NM-32A asynch modules. The serial consoles were VITAL to take over network gear for which the previous administrator could offer no admin credentials. Some of the network gear was also equipment with older operating software, which I had little experience with, so I needed to engage technical help from the equipment vendor. The first two questions were always "Can we do a remote desktop conference?" and "Do you have access to the serial console?" It's always nice to be able to answer "yes" to both questions. There is also some comfort in knowing that Conserver will be logging the session, so you can look back over the logs after the fact to review what happened.

During this session, I was also working with some Juniper gear, and I'm REALLY happy that Juniper also use the RJ45 wiring schema that Cisco uses. I'm a big fan of that symmetrical wiring format.

It's worth noting that SUN Microsystems newer machines (starting back with the Netra T-1) have also used this wiring schema for their TTYs0, and some Lantronix console servers have also used the wiring schema. Opengear has made this wiring schema an option for many of their products, and it is becoming the primary wiring format for some of their newest products. This makes hooking up the majority of the equipment that I work simple, AND it gives me some flexibility to choose different hardware vendors with confidence that they will drop into my existing infrastructure nicely.

Finally, I wanted to point out that serial port capability is also still being built into embedded processors! I've had some time to hack with the ATMEL processors used in the Arduino family of development boards. One port is normally integrated with a bootloader and connected to a USB interface (typically with an FTDI interface chip) to make it easy to talk to the processor. However, these 3.3-to-5 volt "TTL" interfaces can also be tied to an RS-232 interface chip when the ATMEL chip is plugged into an embedded system. (As an alternative, there is another pair of pins which can be a data-only 9600 bps (max) serial port to communicate between the ATMEL chip an another device. And the larger ATMEGA actually has 3 extra serial ports available!) My newest project is to make a small processor which can communicate via a serial interface to another system.

As we enter 2011, the venerable RS-232 serial interface lives on. I wish you all health and happiness, and hopefully a bit of prosperity as well.

-Z-

New info about OpenGear from the USENIX LISA '10 conference

2010-11-13T16:05:00.000-08:00

I'm just back from the USENIX LISA '10 conference (San Jose, CA). OpenGear was there (for their third year), but they were the only Console Server vendor in the hall. As a result, they got a significant amount of mindshare from the attendees. I spent a good deal of time asking questions, and I'm happy to say that they had sent some technically-deep folks to staff the booth. (They also have a lot of Conserver experience, which qualifies for extra points in my book. ;-) I wish more vendors would have come to the LISA conference. With a great international draw, the LISA attendees have significant technical depth and come from some very large corporations. It would seem to be a fertile ground to collect contact info, and provide information to be carried back to universities and big corporations. We'll see you next year in Boston, right?

It's another example of Moore's Law... even in the last year, there are fewer Console Server vendors (Emerson acquired Avocent (who had acquired Cyclades...)), and OpenGear has made many significant improvements in their hardware and software. But, more important to me, is that OpenGear has continued to be attentive and responsive to their customers, not only for me!

What's in it for you, as a Console Server user/customer? Plenty! Customer needs have driven them to provide some really interesting new hardware this year, and they have also given customers the option to use the OpenGear RJ-45 wiring scheme, or order hardware matching the Cisco/Lantronix/Sun symmetrical RJ-45 scheme. Thanks to Jared Mallett and Todd Rychecky for sharing good info with booth visitors, here's some of what I heard;

    They have upgraded the CPU and storage in their smaller devices!
    They have a 4-port model, running on DC power...
    Some smaller units have 2 network connections (dual Ethernet, or Eth and G3 cell modem!)
    They have also made provisions for external FLASH for logging, etc.
    Security improvements include Single Sign-On, and FIPS 140-2 Crypto module
    Participating in the RSA Secured Partner Program SecurID integration

Two weeks ago, I learned of a project that wanted a 4-port device... last week, OpenGear showed me their Dual Ethernet 4-port device, shipping by default with Cisco RJ-45 wiring. Problem solved!

As we approach the holiday season, the daylight gets shorter, and (with the holidays, and folks taking extra time off) the calendar itself seems to get shorter, and deadlines start to loom large. If you need console servers, take a look at OpenGear, and see if you can find what you need quickly.

       Happy holidays!     -Zonker-

Assiging DHCP addresses to APC Power Strips

2010-07-11T14:16:00.000-07:00

This is certainly a post related to serial consoles. But, you are limited in the amount of useful monitoring data that you can pull via the CLI, so folks often want to use SNMP or SYSLOG to monitor these devices. Usually, that's where they want CLI access.

In my case, we are re-numbering the management subnet, and we've decided to try to make this job easier for the next time a re-number is required. We're telling the devices to use DHCP addressing, and using MAC Address reservations for all of the devices that we care about. Two complications arose; How to connect to the serial CLI, and how to set the device to use DHCP.

The complication with using DHCP addressing on the AP-7868 PDU (and likely many other APC devices), is that their default configuration REQUIRES that your DHCP server set option 43 to a specific vendor code. If you don't do this, the PDU will not accept the DHCP Offer from your server.

APC AP-7868 Smart PDU Serial Pinouts

APC RJ-13 serial console, 4-wire, data-only connection
APC Console     Lantronix TS (Cisco, Opengear with Cisco wiring, etc.)
(RJ-13)         (RJ-45)
1  ---(???) nc            default settings:  9600-8-N-1
2  ---(GND)---  5
3  ---(TxD)-->  6         default username:  apc
4  <--(RxD)---  3         default password:  apc
5  ---(GND)---  
6  ---(???) nc 
( nc = No Connection )

NOTE: You must configure a default gateway for these devices (APC-7868), even if you are only talking on your own subnet!

On your Conserver host

This section includes what you need to type on the CLI console, to configure the PDU to ignore Option 43 and accept the DHCP Offer. I use the spare port on one of my console servers to do this, and I always name my unused ports in my Conserver deployments, so that I can use them at a moments notice. In this case, I'm using port 30, on Console Server 5...

NOTE: the CLI is not good about sanitizing user input, or detecting errors! You can type commas (or TEXT!) in an IP address field, and non-standard TLDs in the domain field! Be very careful about your typing, and visually check your data before accepting the changes!

console unused-cs5-30     (this is the port I picked...)
[Enter]
[Enter]     (twice, to wake-up the CLI interface)
apc [Enter]     (default login name: apc  case-sensitive!)
apc [Enter]     (default password: apc  case-sensitive!)
2 [Enter]     (Network)
1 [Enter]     (TCP/IP)
(If you need the MAC address, you'll find it here)
4 [Enter]     (Boot Mode   only if currently set for Manual...)
2 [Enter]     (DHCP Only)
2 [Enter]     (Advanced)
1 [Enter]     (Device Name)
ps-6.12i [Enter]
2 [Enter>     (Domain Name
garage.com [Enter]
8 [Enter]     (DHCP Cookie Is: ...)
1 [Enter]     (Not Required to accept offer)
9 [Enter]     (Accept the pending changes)
[Esc]      (go to the previous menu)
[Esc]      (go to the previous menu)
[Esc]      (go to the previous menu)
4 [Enter]     (logout)

When you log out of the PDU, the device will reload the TCP stack, and in a few minutes, your PDU should accept the new IP address via DHCP.

Configuring your DHCP Server for Option 43

You need to define you option 43. How (or IF) you can do this will depend on your DHCP server software. But, this section includes the particular HEX characters that the APC PDU expects to see (or it will refuse the DHCP Offer).

These clues are for the Internet Software Consortium (ISC) DHCP server. See the dhcp-options man page - this describes all the standard DHCP options. Look for the section titled "VENDOR ENCAPSULATED OPTIONS", which is option 43. If you already have a class for APC units, then that would be a good place to define option 43 as it will be defined for all devices that match the class. The simplest way to define the value is something like this:

option vendor-encapsulated-options 01:04:31:41:50:43;

DHCP option names and numbers are listed in RFC1533, and in the ISC DHCPd source file common/tables.c

(Explanation: I forget the first character (01), but I believe that it designates to expect HEX characters. The second says there are (4) characters in the field. The last three characters are ASCII for "APC". I don't know why the "31" is there...)

NOTE: All APC PDU's should be protected by UPS power.

In an unprotected environment, if the utility power flickers off and on quickly, the onboard network card may not power back on. In these instances, the PDU may power on but the LED display will be blank and any type of access (web, telnet, console) to the PDU will be disabled. In this case, you must power cycle the PDU again, by pulling the power cord, waiting at least 5 seconds, and re-applying power.

The on-board network card was not designed for quick off/on scenarios. Customers protecting their PDU with a UPS, as designed, will not have this problem.

There are also some APC UPS clues on my console site, http://www.conserver.com/consoles/Clues/cons-apc.html

Regards,

-Zonker-

How did the year get by me?

2010-05-30T13:33:00.000-07:00

Wow, I'm surprised that a year has passed since my last post. Largely, it has been due to my inability to get the editing tools to play well with the iPhone and now the iPad. Normally, I don't update my stuff on employer time, or employer equipment. Couple this with a VERY busy year, and me putting updates lower on the priority list, and here we are on Memorial Day weekend.

A lot has happened in the past 12 months. Too much for a single post, but let me sum up.

I'm working with a variety of Service Processors this year, making the work, and documenting what I've learned. Besides working at the BIOS level, I'm trying to determine if/how the settings can be modified from the OS's involved, and that is more difficult than I supposed. The main problem seems like the websites holding the IPMI drivers or libraries have disappeared after 3-4 years. (The biggest lesson here is to capture, collect, and archive these files for any hardware you acquire. Mergers and acquisitions contribute to the loss of the files, even as the instructions make it into a knowledge base.)

An old lesson re-learned, is that older Sun gear will ignore their serial console settings as soon as you plug a keyboard in! Your only clue, if you are logging the console output with Conserver or some other application, is a small line about detecting a keyboard. Removing the keyboard does NOT revert back to serial console output, you need to reboot the server with no keyboard attached. I knew this long ago, but then was thwarted when folks in a remote office attached a keyboard, and we didn't notice that console I/O had gone away.

I've also been doing a LOT of work with SNMP in the past 6 months, and I found an oddity with an MGE (which sold the line to APC, who then sold the line to Schneider Electric) room-sized UPS. The brains of the UPS uses a serial line to pass a data stream to the Network Interface shelf. If this streaming interface is removed, the SNMP cards do not get an update... But the cards simply report the LAST value for any queried OID. As a result, your only clue of a disconnect is seeing all results "flat-line". BUT, if you are also set up forSNMP traps, you will get also see a string that communications was lost with the UPS. This is a case where you really need to be able to parse Syslog and SNMP traps for interesting strings and send an alert.

Other UPS problems led to the integration of a UPS of Last Resort, so that core devices could track and log events with the big UPS. This included watching the serial output of the APC Smart-UPS 2200XL, plus SNMP integration into our monitoring system.

Another Big Project in the past year was due to a planned network outage. This was due to a planned partial power outage in the building. Unfortunately, some key devices also lost power, causing trouble, which could not send alerts via email because the network was down. We decided that we needed to make sure there was a second path out of the data center for critical alerts and messages. This led to integration with a FoxBox, a Network-connected cell-phone device, allowing SMS alerting AND control!

I've also added many more hosts to the host-to adapter database, but I only have a basic Cyclades console connection guide posted, and I still need to make a Digi page. (Cyclades was another fine example of collecting information from the web as soon as you hear about an acquisition!)

I'll make stronger efforts to post more frequently through the rest of 2010. thanks for reading.

-Zonker-

Look at all the pretty lights

2009-05-22T08:26:00.000-07:00

This week, I needed to reconfigure a previously-used Cyclades ACS console server. Finding the right adapters was easy enough, but getting the USB serial dongle to play was the tricky bit. Without a signal tracer, I was stuck with the evidence... "it's not working"...and I was trying different hardware combinations, and a few software changes. I wasted 2.5 hours going through different combinations, and I put a reminder in my calendar to bring my signal tracers and my Serial Doctors Bag the next day.

Once I had my trusty signal tracers, it was a matter of 10 minutes to find the proper USB settings, COM port configuration in my terminal emulator, and start the reconfiguration process. Blinky lights will save you time, every time. Good quality signal tracers are worth a high cost, because they will save you time often, and serve you well for many years. One day, blinky lights will probably save my life...

So, having solved Problem #1, and with kit in hand, I tackled Problem #2;

Years ago, a couple of hundred APC smart power strips (OK, "0-RU, metered Power Distribution Units") were installed in a data center. Fast forward to last week, when one unit is failing to respond to SNMP after a high-temperature 'event' in the room. Since it was in critical service, we wanted to try using the serial console to see if the brains were still alive. Unfortunately, nobody could find a serial cable for the job, and APC doesn't want folks making their own cables so APC doesn't post any signaling clues. I did find one website with clues about the wiring for the 940-0144 cable.

The clues indicated that the 6-pin RJ-14 interface was, indeed, RS-232 (and not TTL, for example), I could do some experimenting. A new toy in my Doctors Bag (not in the pictures (yet), is a MODAPT from Siemon. With this, and my trusty Digital Volt Meter (DVM), I was able to confirm some signalling that wasn't mentioned in the LuxNET web page.

Pin1 -2v (in reference to Pin2)
Pin2 GND (0v potential to Pin5)
Pin3 TxD output (-5.5v in reference to Pin2)
Pin4 RxD input (0.2V in reference to Pin2)
Pin5 GND (0v potential to Pin2)
Pin6 -2v (in reference to Pin2)

With this information, I could then make an adapter cable, and then use the RJ45 signal tracer to confirm that the signals are on the right pins. (I found that I could use just 3 wires, Ground - RxD - TxD) for communications to the APC AP7868 rPDU. I did not need to loop RJ-14 pins 1 and 6, and I didn't need to connect RJ-14 pins 1 or 6 to ground to establish communication.)

I'll update my APC console page and my Signals page soon with some of this information.

APC Serial Ports, and websites, wikis, and backups, oh my!

2009-05-06T22:32:00.000-07:00

How many of you use APC brand UPS gear? Raise your hands! Ok, you can put your hands down.

Now, how many of you have connected the UPS serial port to your computer, to get power information, or to have the UPS tell your computer to shut down? Not so many? How many tried, but had trouble?

Question 1: Were you using the APC-supplied cable (with the correct end plugged into the UPS)?

I ask, because the APC serial ports are unusual! If you use one of the cables you have lying around your shop, there is a good chance that the UPS shut down, and would not turn on again until you disconnected the cable.

When I first tried this, the cable was missing, and I tried various cables in the shop (because I didn't have passive signal tracers at the time). Nothing worked, and I sometimes lost power.

When I had another UPS, with the APC cable, I found that the cable wasn't long enough, and some of the cables I tried using to extend the run caused troubles. That's when I dug deeper into the problem. Here's what I've found;

The APC port puts data and ground on unusual pins!
The UPS uses the two outputs and inputs for other purposes!

You can actually make a "Big Red Switch" for your small APC, using a couple wires and the DE-9 connection. Because of the signals being on unusual pins, when you plug in a PC with a non-APC cable, you are actually applying a "shut-down" hard-wired signal to the UPS.

I made an APC Clues web page, documenting the findings, in late 2007. But, for some reason, I guess I didn't post it. Only when I needed the information again (and couldn't find it) did the panic set in, because some of the legacy information is no longer on the web.

That brings us to....

Question 2: How many of you are making backups of your critical files?

You can replace the word "critical" with Photos, Homework, Research, Family Tree, Tax Info, you get the idea. The data that you do not want to lose in a hard disk crash, or on a lost Flash Memory Drive. Where do you keep it, and how often do you update it?

I use an automated system, backing up critical files to a disk "in the cloud", for $99 a year. This runs every time I start up my computer, and I put a lot of faith in it. But, I need to pay money to keep it alive.

I also make manual copies to a 1 GB external hard disk (but I have to remember to move new files). I should really set this up to run automatically.

For some of my hobbies, and console stuff, I put copies of the web. (Technically, I made them for display on the web, so I guess the copies on my home machine are my backups.)

For some data sets (photos for some of my hobbies or family, or my console-related files, for example) will be occasionally copied to a CD or a DVD, and distributed to friends, family, or colleagues. While it isn't complete, this may be one of my best backup methods!

My point here is that disks die, memory sticks get lost, laptops get stolen, and even servers or hosting companies die without prior notice. Data stored on websites may still be transient. If it is your data, you should make multiple, diverse copies, and make more copies over time as new media becomes popular. (And, if the data is sensitive, ENCRYPT IT!)

When I couldn't find my APC web page (I was certain that I had made one), I started checking the Wacky Wayback Machine to try to find the information. Sometimes this comes up empty, if the domain/pages you seek were not deemed interesting enough.

In closing, if you have any APC UPS gear, please check out my APC Clues page. Save a printed copy, because you may want that information again someday. ;)

Regards,

-Z-

How about Ubiquitous USB Serial Consoles

2009-02-20T17:20:00.000-08:00

Happy New Year! Sorry I've missed posting in a couple months.

It's odd to me, when sysadmins tell me that "Serial Consoles are Dead. USB killed them." But, when I ask them what they mean, I get different answers.

The first meaning is: "RS-232 is dead, long live USB!" So, then I ask them what they do when they need to get to the serial console, and they tell me about needing to find a USB-to-DE9 serial adapter, get that to work, and point the serial console getty to that device. ("I bet that's tough to do when you are having trouble with a server" I offer. They usually agree that it is...)

The second meaning is: "You don't need DE-9 connectors anymore, you can do it all with USB". This has more insidious meanings...

2a) Most new servers don't have a DE-9 connector on the chassis, just USB ports.

2b) If I need a serial port, I can add a USB-to-DE9 dongle, and use minicom.

Have you ever tried plugging in a USB-connected external modem, and then tried to dial into it, to access the serial console (getty) on your server? The adventure is fraught with problems, which must be solved with a Crash Cart or other form of KVM or remote desktop (which means you need to think about it BEFORE you have a problem, so that it's a useful avenue when trouble presents itself).

Remember, the serial console has been your friend, when the network or the network-facing configuration goes awry. Do you think USB is ready to replace that role? Can you go to Fry's, and buy a USB-to-USB cable which you can simply plug in between two servers, and automatically establish a "null-modem"-type data path? I haven't seen one. Why Not?

Someone needs to develop a little, microprocessor pod, with two USB cables. Power the unit from the 5-volt signal from either USB device. Maybe even add some TX and RX data status LEDs, to show which USB cable is talking. You don't even need to do the conversion from TTL to RS-232 and back...just keep it all at TTL levels! But the microprocessor needs to be able to talk to each USB port when the computer inquires "Who are you, what are you?"...this is the key!

USB relies on drivers, residing on your computer(s), to recognize the ID sent from a USB-attached device when you first plug it in. To make things easy, many basic drivers are bundled into Operating System packages, which lets you plug in many devices without needing to install the drivers first. (This is the difference between Play and Pray in "Plug-n-Play" architecture.)

Looking at the basic USB-to-DE9 dongles, most are derivative works from two basic hardware types. For example, since Brands A, G, M and Q all use "Chip X", and Brands D, I and S all use "Chip Z", it's easy to see that just having two basic (generic) drivers will let you plug in most of the devices from these 7 brands of dongles. Getting Microsoft, and the various Linux distributions is an important step in making things easier for the folks who use these operating systems with serial devices.

But, if you make this USB cable I'm describing, and you try to use either of these USB-to-Serial chips as a base, you will find that most systems will think that you are attaching a plain serial device, and they'll use the generic driver. So, you'll need a bit more engineering...you need a driver that will determine if it needs to find (or create) a getty process, and connect it to the I/O address for the cable. Since you are going to need to make your own driver, you just gave yourself the flexibility to define how this type of cable communicates with the computers.

In a basic "Crash Cart" scenario, I want to plug the "Server" cable into my server chassis. If there is a console getty process running, I want it to be directed to this new device (even if it thinks it was connected to some other previous USB-to-Serial dongle, since that may have failed - causing me to need the crash cart).

When I plug in the "Client" cable to my crash cart, or a laptop, I want them to think that it's just another serial port. (For real Ease Of Use, let's default the serial speed to 9600 8N1.) If it's a Windows box, I'd like the option to choose (in the driver configuration) which COM port this device is assigned to (so it's going to ALWAYS be there...even if it needs to steal the assignment from a previous device). I'd also like to be able to associate this interface with an application, so that the OS will try to launch that application when the cable is plugged in.

I want to walk up to a server, just out of the box, with my laptop, and plug this cable into the server and into the laptop...and have the laptop launch my copy of ProCom, or TerraTerm, or HyperTerm...and have the server configure it's getty to talk to the new port, if the OS is healthy.

If we can loop in a few of the big BIOS makers, the BIOS might notice this device on startup, if BIOS redirection has been turned on, and use the cable (as a preference above the usual COM A assignment).

Of course, until we can get to this point, USB hasn't replaced the venerable RS-232 serial port, at least in my data center.

Good luck with this winter!

-Z-

Why aren't service processors "easy to use"?

2008-11-30T23:13:00.000-08:00

You could try to tell me that it's because serial consoles are sensitive access points to vital hardware, and that Security is inversely proportional to Ease Of Use, therefore Service Processors cannot be Easy, because they need to be Secure. And, doing my best Jeremy Clarkson (Top Gear) impression, I'd tell you that "you'd be wrong".

I'll tell you that it's because most the consumers are not asking for ease of use, or for interoperability. Because it seems (to me) to be on the minds (and mouths, and blogs) of a relatively small subset of vocal customers, and the 'collective voice' of this subset is further diminished because we are asking different vendors. (That is, I ask Dell, you ask HP, she asks Sun...but darn few of us are asking more than one or two vendors.)

Security features are only just now beginning to be built-in, either already enabled, or very easy to configure, largely because the US government has tied such requirements to their purchasing rules. If you want to sell the US government a desktop computer, there is now a whole checklist of security items to tick on the list before your kit might be considered. The US government, across many departments, now speaks with a large, unified voice (through the purchasing rules and policies)

Similarly, in the early days of the Point-to-Point Protocol (the follow-on to S.L.I.P. for IP-over-modem communications), PPP would allow more computer network protocols to connect and exchange data. As a customer, you'd have thought that vendor A's PPP (the choice inYOUR shop) would talk to Vendor B's gear at some Strategic Partner's network. But you'd have been wrong. Coupled with the birth of the Commercial Internet (it wasn't just for DARPA anymore), many companies were trying to get onto the Internet, or to connect to other companies to exchange information.

There was a large dissatisfacton with a lack up interoperability. And many companies spoke with the vendors to complain. BUT, besides complaining to the vendor who supplied the kit to YOU, you also would have complained to the vendors of the other guys gear as well. You might complain to 5 or 10 vendors, between routers and modems.

As a result of the dissatisfaction of LOTS of users, 6 companies started the PPP Consortium, with a week-long meeting at Telebit Corp in Sunnyvale, CA, with the express purpose of trying to test all of their gear with each other, and test across the matrix of then-new (and still evolving) PPP protocol options. At then end of it, the scorecard wasn't too bad, and all of the participants had a list of things to fix. They also hashed out some of the ambiguities in the draft Request For Comments (RFCs) describing the options. (Many of the vendors were part of the IEEE, and were helping define the RFCs, but the meeting helped settle misunderstandings about the options.)

Because the group of six could now claim a large amount of interoperability with the others in the group, it helped their marketing. Because the customers saw progress, they harped on the vendors who were NOT part of the original group, and they wanted to get in on the consortium. Six months after the first group, I believe there were 13 members, and 18 at the next. By the third meeting, PPP was Stable. PPP 'just worked', most of the time. PPP had become "easy", and SLIP was largely relagated to antique links on old equipment.

When I look at the various implementations of BIOS redirection to the serial port, they almost exclusively presume (or require) a modem connection, which then requires hardware level wiring issues if you are using a Console Server connection. The mapping of Function Keys, command sequences, and the timing issues to invoke the commands make using them tricky on a Console Server. It makes them impractical on any link slower than 19.2 Kbps. Yet the few BIOS makers don't see a need to try to do better, let alone to work with the other vendors to provide some consistancy of user interface to their customers. Do they know whether you care?

Now that Service Processors are becoming prevalent on new servers, it's hard to get a server without one. Yet they suffer from problems very similar to BIOS. The user interfaces are different from vendor to vendor. But what about processors from the same vendor. What about within the same vendor, across a hardware platform? You'd think that a vendor might want to keep their command sequences consistent, for the sake of their customers (who might be scripting support sequences for their machines). You'd probably like to think that...but you'd be wrong. A couple vendors seem to change at will, but I'm guessing that they are just buying server designs from other companies...and those same vendors probably haven't taken the time to design that Human User Interface, they probably simply rely on the hardware maker to fulfill the tasks (power cycle the server, set the IP address of the Ethernet interface for service processor, implement a web interface with certain features), leaving the details of keystrokes to the hardware maker...and when the vendor chooses another hardware maker, they get the same latitude with the design, and make different choices. (I wonder what Donald Norman thinks about tis problem.) Do your vendors know if you care?

When you tell a sales person that you want some new feature on their next year's model, the usual response is "OK, if we can get that done, how many will you buy?". It's going to cost them labor hours, engineering and QA time, technical documentation, maybe changing part of a training class. While you may only want 6, or 12, or 40, your voice will be tallied. When they from enough of you, it's going to be worth it to make a change. But we need to tip that scale by asking, and probably tying that request to some money. (My dad would sometimes ask me to "write your request on a $5 bill, and I'll consider it", but I have seen this over and over again, from different seats in different companies. For the vendor, it's all about money.)

My point here, is that you are going to spend money on new servers at some point. Make your money count! Tie the money to your requests for easier-to-use service processors. Decide what you want, and tell your possible vendors about your requirements. They all want your money...so vote with your wallet! Reward the responsive vendors! If the unresponsive vendors don't get the order, they'll at least know why, and they'll learn that you were serious. As more of us tie our requirements to purchasing, we can make a difference.

As you get into your busy end-of-year, some of you are already going through your budget cycle for 2009. Use any slowdown you get to think about requirements to go along with those purchases. Get them on paper, and in email, so they are ready to send to your vendors with your Request for Quote. And good luck in the new year.

-Z-

A little perl of wisdom

2008-10-20T11:04:00.000-07:00

When it comes to finding some quick help (after the Helpdesk has closed), or those tidbits of knowledge and experience of someone who has tread the path before me, Google has been my friend for many years (taking up the mantle after AltaVista!).

But, there is a downside to learning all about a topic from Google. It is that you, the reader, are guiding the learning. You are getting the answers to the questions you ask...but it doesn't always offer things that you should know (or at least consider), based on your searches.

I taught myself perl, reading books, and using Google for answers, and examples...but I sometimes found myself confused when reading someone else's code. This year, I'm taking formal classes, from a few sources, hoping to patch a few holes in my foundation of knowledge.

While I'm glad to have taken this approach, to have someone guiding me through the learning, systematically, and answering my questions...I'm finding that the Teacher in this equation is even more important than the content. Even if someone knows the content well, that doesn't mean that they will be a good teacher. (Answering the occasional question does not equal teaching.)

After 5 weeks in a semester-long class at a local community college, I too a full-day Perl 101 course from Bayview Training...and then I coded for 15 hours straight, because that class had made a number of concepts more concrete for me (several "light-bulb" moments), and I was excited about the possibilities for my future projects.

I heartily recommend Bill Ward as an instructor. While he teaches primarily in Silicon Valley, he does travel for some classes and consulting as well. The cost of the course was money well spent! If you really need to learn some perl, check out Bayview Training and Bill Ward!

Big News! SSH clients for iPhone!

2008-08-28T17:52:00.000-07:00

I haven't had any 'big news' on the console front for a couple months now. I have been missing my BlueConsole adapters (which I've blogged about before), and SSH access from my iPhone, BUT that has changed!

New in the Apple Apps Store are four SSH applications (use the search dialogue, look for "ssh"...). I've already bought two to try, and one has already met my most common needs. In a nutshell, here are the four I found, with URLs to the maker's websites, in case you want to do your own homework on the topic.

TouchTerm http://www.jbrink.net/touchterm/index.html $2.99 This one looks pretty good, from the online docs. There is only basic information on the maker's home page, BUT, check the SUPPORT page! Key Exchange, font colors, different input modes (immediate, batch, command history). The developer offers an email address for comments and suggestions. This looks like a good pick!

pTerm http://www.instantcocoa.com/products/pTerm/ $4.99 This maker also lists an email address, as well as a Google discussion group. The web page posts a short-term development schedule. They also boast that this app is based on PuTTY, and I'm a big PuTTY fan (but I understand that this app is only scratching the surface on features). I've sent email to the developer with my wants, and I already received a positive reply. He is also willing to work on the bluetooth-serial-devices interfacing as well.

iSSH http://www.zinger-soft.com/iSSH_features.html $4.99 This one also looks good. The main web page is odd... there are small, light-grey arrows under a few of the descriptions...they indicate that there are additional items at this menu level of the web page. Clicking on them will show you the other topics. This one also supports shared keys, but doesn't seem as flexible (to me, yet), as TouchTerm.

SSH http://www.throughput.biz/ $3.99 This looks simple, and it word-wraps the screen. There is little information on the maker's homepage. I'm currently not inclined to try it.

So, with this in mind, I have bought the first two (TouchTerm and pTerm), and I'm spending my efforts to put TouchTerm through it's paces first, because pTerm doesn't have RSA/DSA keys yet (though keys are on the short-range list for new features, so I'll be giving them both a good workout in September). I was able to set up a basic profile, with screen and font colors, RSA-2048 key, emailed the key to another host, so I could place the key, and use the session. It will take a bit of getting used to, since I'll need to leave the fonts a bit larger, and pan around the screen (using mutt was kinda awkward, but that won't be my primary use of SSH). I like it so far, and I hope to be able to put my BlueConsole adapters back into service sometime in the near future.

I hope you had a chance to enjoy some vacation this summer as well.

-Z-

Social Networking, professional connections

2008-07-13T04:17:00.000-07:00

The question comes up, from time to time... Why do System Administrators seem to be better at (Social) Networking than Network Administrators. You'd think that Netadmins would have the communications thing down cold. But, I've found, we tend to have smaller social networks, but with links we can rely upon to do our jobs.

Linked in? Yep, I've been there a while...

But, why are there so many more Systems-related conferences than Network-related? OS related trade shows, versus Network trade shows.

I just find it odd, that's all.

-Z-

Can you hear me now? Wait, now I can't hear you!

2008-07-04T15:36:00.000-07:00

California's two new laws regarding hands-free cellphone use while driving have Bluetooth headsets in the news, and on the front page of many papers. I still haven't found the right combination in any solution that I've tried. Every one has had some deficit.

Since I wear glasses, I don't like anything that has an ear-loop. None have been comfortable so far. I also prefer an in-ear (ear-bud) style, versus an over-the-ear solution with a headband. This certainly limits my choices, but there have been a few good choices. Good, but none were Great.

For listening to calls, the best for me has ben the Sony HB-808 headset. Good talk-time, and lightweight, with sufficient audio to the earpiece, and the microphone has some automatic gain control, but it is not so good about noise cancellation. It comes with an earloop, which can be reversed to wear on either ear (that's fairly standard), but the art near my ear was also uncomfortable, because it pressed against my ear when I used the earloop. But, the diameter of the earphone was slightly smaller than an original iPhone earphone, so I tried using the Griffin EarJam earbud adapters for the iPod earphones...This was a HUGE improvement! It now fits in my ear without an earloop. It's light enough to stay in my ear. And, because the earbud is in my ear, it seals out local noises, while allowing me to turn the earpiece volume down! Even in a data center, staring down dozens of fans, I can hear my calls just fine...but, that's where it all breaks down. Because the microphone has gain control, it hears all of the fans, so I constantly have to juggle the phone to mute and unmute the microphone during calls in the data center to service gear. It frustrating for me.

I finally bought an original Jawbone, because of the noise canceling technology. It really is great, for the people I call. I don't have to mute my microphone in the data center (in most cases), which has been really useful. But, the device has an awkward earloop, since the device MUST rest on your cheek in order for the noise canceling to work. And, since it's pressing the microphone aginst my cheek, the earpiece hovers above my ear. Folks can ear me, but I've got to turn the volume up to 11 to hear them over the local noise.

I tried all 4 of the ear-bits that came with the Jawbone, but none were comfortable, and only one came close to helping me lower the volume of the earpiece. The earpiece is also slightly smaller than the iPod earphone, so the Griffin earbud trick wouldn't seat securely. Even trying to secure the Griffin part, it would fit in my ear, but then the microphone wouldn't press against my cheek (defeating the noise cancellation), so that trick didn't work. BUT, at MacWorld SF 2007, I found Comply's "Whoomp!" earbud enhancers. They are snugger, and more flexible than the Griffin part, with a foam earpiece on a slightly slanted axis. This held the Jawbone snugly, pressed the mirophone against my cheek, and sealed the outside noise nicely...but, the foam bit was hard to clean. I still use it, but each earbud only lasts about 3-4 months for me.

A friend turned me on to another option for the original Jawbone; Using the Jabra Ear-gels (available from Hello Direct for about $6US), which gives you another set of 4 sizes, which SNUGLY fit the Jawbone. hey are more supple than the original Jawbone ear-bits, but they still aren't as comfortable as I would like. My friend has already tried the new Jawbone, and has been favorably impressed.

I've tried using commercial adapters (cable, switch and microphone) with my own iPod earbuds, but the microphone still hears the fans. (It's a dichotomy; If I can hear you well, you can't hear me well...why can't they make a headset that allows both?)

I'm fine using a headset in the car, but I received a Tom-Tom for my birthday, since it has bluetooth hands-free speaker phone capability. This works OK, but I understand it's a high-theft-rate target. I can't leave it in the car, lest someone breaks a window to steal it. (Even if you take it inside, I'm told that leaving the suction-cup mount on your windshield is enough attraction that a thief may smash your window to see if you stashed it inside the car.) As a result, I now have one more thing to stow and carry back and forth.

Should I try using the Tom-Tom as a speakerphone in the data center? They don't brag about noise cancellation, but folks seem to hear me pretty good when I'm in the car. :-)

-Z-

Zonker needs a Bluetooth terminal emulator App for iPhone

2008-06-05T23:05:00.000-07:00

The longer I go without this capability, the stronger my desire for an application for the iPhone to fill my needs. But, I don't want to hack my phone...I want a real iPhone App, and now it's worth money to me. I can't find a 'Wish List' page that hasn't been taken over by spammers. So, I'll try the Open Letter approach.

I have two needs, with a common root...both need a basic vt-100 terminal emulator underneath. ANSI color would be a nice touch, but isn't necessary. I don't need Wyse, or full-blown TN-3270 emulation...keep it simple. Some scrollback buffer would be useful (say ~2600 lines), and the ability to capture to a log file (and email the log later?) would be very handy. So, what else?

The primary need is to have an SSH v2 client, so I can SSH directly to my machines using the Edge or wifi connections. This also probably means solving the pre-shared keys issue...how do I get the keys onto the iPhone? I still can't sync a simple file to my phone. Maybe I need a key-generator on the phone, but I still need to get the public key to the hosts.

The secondary need is to communicate with a Bluetooth serial device. Specifically, a BlueConsole dongle, although there are other devices with which you should be able to interoperate. (I used to use the Tri-connect package (from Tridone) on my Treo 650. It wasn't great, but it was a good option, which let me do a lot of quick, little jobs without lugging a computer around.) I'm sorely tempted to carry the Treo around in my tool bag, for those occasions when I'd really like to use these serial devices...if I could find a good way to keep the Treo battery charged while it's in the tool bag, I'd do it. But, I'd much rather use the iPhone for this task.

If you're an iPhone developer, working on a terminal emulator that will use the Bluetooth port, let's talk! I'm willing to be a software tester, give feedback, and help with documentation. I've got good credentials in both the serial console and the communications software test realms. We may also be able to work out a loan of a BlueConsole device while you're developing.

If you're an iPhone developer, working on an SSH v2 client, I want to talk to you as well, and I'm willing to be a tester for this as well. (I'd love to know that someone is trying to fulfill this need.) My favorite SSH app is PuTTY (by Simon Tatham and others), if you want to set your sites high. If you are trying to determine what folks 'want', versus what the 'need' (and would be willing to pay for), I'd like to talk to you.

Since Father's Day is in June, I'm going to post information about some of my favorite new tools in my next posts, in case you are shopping for special gifts. (Yes, the iPhone is on the list.)

-Z-

What is 'Normal', anyway?

2008-05-13T09:41:00.000-07:00

Yow! It's already well into May, and April got by without a post. It was a busy month, with a LOT of discovery tasks happening. I'll try to get a few more posts in this month to summarize. But first, I wanted to get this thought online...

Those who know me are not surprised if the come into a dark wiring closet, and find me sitting on a milk crate, staring at the 'blinkly lights'. (Frankly, I've startled plenty of people who don't know me this way!) Most just write it of as a fixation on blinky lights, but it is not. This is how I can 'visually spot trends' of activity on devices and interfaces. It's also a chance to remember which link status indicators are on, off, in an alarm condition. And, I'm also listening... to fans, air conditioner inlets, hard drive spindle bearings, modem speakers, relays clicking.

This is how I get to know what "Normal" looks like, sounds like, and smells like in the data center and wiring closets. (Yes, smell... does it 'usually' smell damp in this room? Is the 'ozone' smell something that's always here, or does it indicate a component failure. Scent is a strong trigger for memory!)

Knowing what is 'Normal' helps us spot what is unusual. When I have a network failure, I can go to the associated wiring closet and look, listen, and smell...I don't need to ponder "has that always been like this?", because I'll remember. "That light is usually blinking...so there isn't traffic on that interface!" Knowing what is normal is a key to fast troubleshooting.

The RRD tool has been a great resource for graphing monitoring data, allowing you to visualize 'normal', to see 'now', and to identify trends. You find this under MANY open source tools, such as MRTG, Cricket, Cacti, and many more.

But, can you tell what's 'normal' with a serial console? Yes, you can! The key is, you need to LOOK when things are operating normally. Look when the system/device/network is idle some night or weekend. Look again when backups are running. Look again when the network is busy, but not failing. Then, compare your notes, or, your LOGS!

You can do a LOT interactively, using a simple terminal emulator and a cable. You have a lot more flexibility with a Console Server and multiple Telnet sessions (for example, you can monitor many consoles simultaneously, and cut-and-paste between them). The real benefits are had when you combine the console servers with a Serial Console Management Application such as Conserver, or ConsoleWorks, and you can compare historic data with today's results.

I have a handful of devices which have "diagnostics' ports, that only the field engineers will use. They are not for normal use by customers. However, when you connect to these ports, you can find some of the devices are 'beaconing' about events, or reporting regular status messages. Even if the port doesn't say much, if you hit a carriage return, you'll probably get a prompt...doing this occasionally will tell you it's still alive. And, sometimes, you can also leverage the simple diagnostics that are built in. It's good to have a baseline log of what the devices are reporting to these diagnostics when they are healthy, so you can compare them if the device starts misbehaving.

But, to do any of these things, you need to start by looking when things are working as expected, doing their job the way you want them to. That's when you want to get your first look, and save that data for later comparison. (Speaking of comparing log data, try SPLUNK! Check out splunkbase, and the SPLUNK Forums! Splunk is worth a few blog articles by itself...later.)

The bottom line: Do you KNOW what you are missing? Do you WANT to know? Then LOOK!

-Z-

One moment please, I'll connect you...

2008-03-31T21:58:00.000-07:00

Today's topic has been a recurring theme during the past few weeks.

On the one hand, while I'm in love with my iPhone for email and web access pretty much everywhere, I still really miss having an SSH client on my hip. (On my Treo, the pssh client did the trick. It was simple, talked SSH v2, had a poor random number generator, but could import strong pre-shared keys from a Palm memo.) I'm hoping that a simple, cool new SSH terminal app for the iPhone is just around the corner, now that the iPhone Software Developer Kit is out. I don't need fancy key-mapping, or a variety of terminal emulations. If it can offer to let me talk to Bluetooth devices as well as the Edge/WiFi connections (so I can use my Blue Console devices, it will be worth more to me. (That's a blatant plea/hint to any iPhone developers. Hopefully I've said iPhone enough times for this article to turn up in a web search. :-)

On the other hand, I've found the Service Processor on some SUN Opteron boxes to be a bit vexing. Many hosts have these Service Processors (S.P. from here on) on-board now, and each vendor is implementing it a bit differently. It's an added 'black box' on the communications line between the physical serial port and the serial I/O (tty) port on the host.

When BIOS Redirection was added, this was the first complication making serial ports provide the GETTY console login under a Linux host. Now these processors, supporting ILOM/ALOM/e-i-e-i-o communications are also listening to the line, looking for meta-sequences, waiting to interrupt the conversation and do your bidding. One gotcha is, when you start talking to the S.P., you lose any logging from the Operating System. (Why don't these special purpose computers BUFFER at least SOME amount of host data?)

The odd hang-up with the SUN units I'm wrestling seems to be an inactivity timeout. If I get into the S.P., and let it sit for a while, it seems to go to sleep, and I can't wake it up again. BUT, it never returned the connection back to the OS. Essentially, the O.S. console connection is now bottled-up, and I can't get the S.P. to answer through the serial port, or to relinquish the port back to the O.S., leaving me to reboot the system in order to get the serial console back. There MUST be a better way.

Fortunately, there is a good write-up on the SUN S.P., and I'm going to go over it this weekend. Hopefully I'll find some variables to change the meta characters which trigger the S.P., since they are too close to another sequence I need to send, and I also hope to find some timer settings to get the S.P. to drop out politely if I time out of the session.

-Z-

Tools and trade-offs...

2008-03-01T23:40:00.000-08:00

Recent circumstances gave me the opportunity to switch my phone service from a Palm Treo 650 to an Apple iPhone. I want to ignore the OS/platform discussion, and the monthly service costs, at least for a moment, and look at the move from a functionality standpoint, because it's yet another example of the trade-offs that we need to make while working with the technologies of our choosing.

Before I used the Treo, I'd had a some type of Palm device for many years. It held simple notes in memo files, and larger, illustrated documents were composed and viewed with TealDoc and TealPaint (from TealPoint software). I could 'beam' (InfraRed data transfer) information to other nearby folks with Palm devices to share data, or send the files from a desktop computer by email, or post them to a website to share them with folks farther away (in time or space). But I was happy carrying a separate cell phone, because I hadn't seen a really GOOD marriage of the two technologies yet, and I didn't want over-using one device to drain a battery to disable BOTH functions (PDA and Phone).

I even received a Treo as a gift, but I gave it away. Then, the recipient found a good VPN client, and an SSH tool tat used the data service of the phone...THAT was a feature I would find VERY useful! Later, I found the BlueConsole devices, and the TriConnect software, making the tool even more valuable. I used the Treo for two years.

The iPhone doesn't have a good terminal emulator (so far) that will let me talk to the BlueConsole devices. I can't find a good SSH tool, either. So, you might ask, "why would you want to switch?"

Because, for over a decade, I've wanted a simple, fast way to check email, AND the web. Initially, I wanted this at home, since I usually had fast access at work. But there were sometimes moments when I was away from home when I'd remember I wanted to send email to someone, or I wanted to look up something on the web. When this happened, I wrote an appointment on my Palm, to remind me to send an email, or check something. I'd have to set the time for the next time I thought I'd be near an operating computer, which was sometimes a couple days.

In some of my earlier days, I'd used X-terminals, and other thin-client devices, which booted quickly. But I don't have any of that hardware in the lab. I'd thought about using a simple terminal, and a console server with an "auto-telnet" feature, but that would only give me a quick access to command-line email. All of the options were lacking.

So, the iPhone uses the phone (AT&T Edge network) and WiFi for data connections. It easily remembers WiFi connections where you need to authenticate, and uses the phone when WiFi isn't available. The Safari browser isn't "perfect", but it is so much better than the Treo had been. And the built-in email was now reasonable to us, because I had gigabytes of fast "in-phone" storage. (Many Palm apps didn't play well if you were running them from the SD card.)

So, I have retired the Treo to being a BlueConsole terminal and file repository, since it does those better than the iPhone. But I don't carry the Treo on my belt anymore.

Now that I've used the iPhone, I've decided it was the RIGHT answer for me. I'm REALLY glad the phone doesn't tell folks where I was when I've composed/sent my email. But reading my email has replaced reading my daily CBC/BBC world news (I used to get them via AvantGo, since browsing the web wasn't practical on the Treo.) And, because I'm reading my personal email during the day, it has given me back some time in the evenings. And, being able to send and respond to email more quickly has also sped up some communications exchanges.

The web browsing aspect has also worked out very well, because of that finger-pinching technology that allows you to shrink or expand your view. I need reading glasses now, and some web pages are really fond of the H5 and H6 'fine print' header styles! It's not a problem for me anymore.

I've also put some songs on it, and left my iPod at home. And I also put some pictures on it. Oh, and the built-in camera is pretty good, too. But these weren't really on my radar for making the decision to switch.

I miss having an SSH application in my pocket, and I hope I'll get a terminal emulator that speaks Bluetooth someday, but I've grown too attached to my email and browser to go back to the Treo. The chances are good that applications will come in the future.

We all have to make trade-offs. Here's hoping you are happy with the trade-offs that you need to make.

Bluetooth console adapters, short time only

2008-01-06T22:47:00.000-08:00

I thought I had mentioned these before, but I can't find them by keywords, so here we go.

If you know about the BlueConsole2 bluetooth serial adapters, and you wanted to buy any, order them NOW, since the manufacturer is closing his doors this month (JAN 2008). The ordering page on his site is already closed, but he still has stock, and is trying to make deals to move the last few hundred.

I waited 6 months, trying to get someone else to buy them, so I could try them. OK, now I'm the guy that bought them. They are a bit pricey, but the are clearly the best implementation in this niche product space.

I'm using mine with the TriDone TriConnect software on my Palm Treo 650. The free version of the software throws an extra carriage return (so I'll probably upgrade to the Pro version), but it's a great console with scrollback for watching a device start up, and doing some CLI configuration.)

There are plenty of times where I don't have a laptop handy when I might want one. This tool is an investment in my time. When I need a terminal quickly, unplanned, I'll be able to use the Treo and the BlueConsole.

If you want to save your time, then don't waste any time. Get yours today, and save yourself the prie of trying to buy one on ebay later.

-Z-

It's OK to talk back...

2008-01-03T12:52:00.001-08:00

With the new year came some new toys, but I still have the same old Doctor's Bag, so I needed to find out what needed to stay, and what could be combined or made smaller.

One of the relatively large things were a couple of DB25 Loopback modules, from Cyclades (one included with each server, so I have a bunch). To use this useful tool, I also have a collection of DB25 adapters for a few of the console servers I often work with.

I'd tried using some simple loopback plugs...just short wires into an RJ45 plug (sometimes called an 'ice cube') like the one you also get with a new Cyclades console server...but these get lost in places, sometimes even lost in console server ports if bigger gear is mounted above and below the console server, so I gave up on them.

My next attempt was a 'pigtail'...a 4"-6" section of cable into the 'ice cube'. This was a 'flag', to help me find it sticking out of the console server, but I still had the trouble getting these out of console servers which had been mounted in tight corners.

All the while, I still needed a way to check the signal at the other end of the cable. My real concern was usually "do I have good connectivity up to the back of the machine?", and this required a loopback at the device in question, so I started packing the loopback modules.

I'd thought about taking an RJ45 socket out of a patch-panel, and looping wires around in that. It would be smaller and lighter, but how do I label them. I thought about using color, like the hood colors on the console server adapters. (As baby boomers get older, color coding lets one get by without reading glasses, at least some of the time.) I'd tried a variety of discarded sockets, with some success, but they all had flaws.

I finally stumbled on a part that I like for the purpose! Panduit's Minicom line of CAT-3 patch panel parts. You can get them in the common resistor color-code colors, they are compact, rugged, you can buy them in small quantites, and they have a flat spot you can use for labeling the jack. I'll try to get a couple pictures on Flickr, so I can link them here (and update my Doctor's Bag page). The only real modification to make is to round the sharp corners with some sandpaper, so they don't cause problems in the bag.

-Z-

Happy New Year! It's about time...

2007-12-30T09:57:00.000-08:00

Time for a New Years Resolution! Establish an NTP server at every site I work on, and point as many devices to it as I can. (I'll do this to save everyone some time.)

Consider taking a look around your installation. How long would it take to check the uptimes of the devices, and also check their on-board Real Time Clocks. How many do you think are correctly showing the current time, within +/- 10 minutes? For your network gear, how many are showing log timestamps as the "from startup" uptime stamp, instead of a date/time notation?

Now, consider that, in the time it would take to EVALUATE your clocks and timestamp settings, that you could probably FIX THEM, not just record the results! If you plan to go and touch all the gear anyway, why not save some time, and SET the clocks, instead of checking them?

But, when was the last time anyone bothered to do this, in your installation? Maybe the clocks on your devices only get set when each device is installed. (If you ship devices between data centers, across timezones, do you make a point of changing the device clocks when you receive them into your installation?) Let's face it, these clocks are cheap, and they ALL have some drift in them. Even if the drift is minimal, the error is cumulative, and can be significant across 3 or more years. You complicate the drift when you don't use a stable starting point. (What time reference does everyone use when they set device clocks? Probably their independent wrist watch?)

In the same way that you need to spend some money to make more money, you need to spend time to make more time! NTP is a great way to set clocks, and keep them close to the current time. (Even a daily time sync will all but eliminate the effect of drift on the cheap clocks.) Most devices have a no-cost client available, and many servers and some network gear have a no-cost NTP Server option available. If you have a complicated network, with multiple customers or DMZs, you can invest in an appliance that uses GPS or the cellular phne service to provide a drop-in connection on an isolated network. You just need to take some time to set up a server, and then set your devices to be clients.

Technically, all of this work (and invested time setting up NTP) doesn't save time, per se. What it will do is reduce the time you spend tracking down problems. Calculating time-math is hard. Correlating time offsets in log files between devices with unsynchronized clocks takes a LOT of time. But, when you have an "event" in your installation, and you need to find the Root Cause, you need to compare a bunch of logs, to figure out what happend first, then second, and etc.

If you take the time to set up an NTP server, you make it quick and easy to 'set and forget' the clocks on new devices coming into your installation. When you take the time to point all of your existing devices to the NTP server, you'll save time comparing the logs on all future 'events'. It's hard to guess how much time, or how soon you will recover that time, but it WILL happen. Just give it time.

-Zonker-

Some NTP appliance resources I've found to be interesting:
VMware NTP appliance VM info
Symmetricom S200
J-Time (Meinberg USA) Lantime M300
Brandywine Communications NTV-100RG

Saving time, and saving soles...

2007-12-28T22:55:00.000-08:00

Whether you wear sports shoes, cowboy boots, or birkenstocks, the life of your shoe's soles are determined partly by how much walking you do in them. I'm a big fan of recreational walking, but I'd like to minimize the running around I need to do at the shop.

In the event of an emergency, getting on a serial console saves me from pushing a cart around, waiting for elevators, carrying a laptop around (and all the extra adapters, cables and power packs), just so I can get on the console of various devices, just so I can check configurations.

Yes, if the network is working, I can check a lot of the configurations across the network. But, sometimes interfaces die. Sometimes a typographic error will change an interface or network setting. Sometimes a cable gets unplugged accidentally. That's when your normal access breaks, and leaves you scrambling. Unless you have a console management network in place, and some strategicly placed console servers. (Better still, you should have something logging all of those remote consoles, but that's a topic for another post.)

Today I watched a friend cutting over to a new network over the holidays. Progress was being made, but it was slower than planned, and the days are starting to run out. He had developers from the new network gear vendor helping to debug the lack of interaction. While they all had their laptops, I watched as they wheeled a cart around the buildings, and up and down elevators, trying to check configurations because the network wasn't yet stable.

What was missing was a stable, simple console server deployment, with 8-16 ports in each of the main and intermediate frame rooms (MDF and IDFs). There was already fiber between the rooms. The console net could have been simple, stable, and independent of the main network. And it would have allowed them to be logged into many consoles at one time. They could have been watching errors and events on many devices as they tried tuning various settings.

Sure, this costs a bit of money to set up. But the price per port is low for simple, reliable gear. Consider the time for a couple contractors, and three developers, working over the holidays, trying to debug a problem. (I guess if your doing that work as an hourly worker, it's not so bad...but if you're the person in finance trying to close out the end-of-year books, and the network is being flaky, I imagine that your perspective about how soon the network should be stable would be different.)

I've written elsewhere about my portable Emergency Kit (a small hub, console server, adapters, cables, and canned telnet configurations to the console server). Today, I'm trying to lobby for you to consider a simple configuration, to support your current devices, with a bit of room to grow. Hopefully, you may find yourself trying to add more gear in preparation for a cutover, adding extra devices in every wiring closet, and you'll save yourself time and steps if you have some extra ports ready in each location.

Remote offices/sites deserve this consideration as well. I know many places that install a modem on the core router at their remote sites. But, what happens when the router relies on a TACACS or RADIUS server at that location, and the problem is not the router? You can dial in, get a prompt, but you can't authenticate, so you can't get into the router, or the authentication server. If you had a small console server there, and the modem allowed you into the console server, you would have a better chance of getting the access you need. (Even if you only see errors, you'd understand how to resolve the problems later...maybe the authentication server is trying to log your attempt by resolving to a DNS host that it cannot reach (since the network to the main office is down)? You know you need a DNS host in that office, or at least some static entries in the local hosts table for that host.)

Trust me about this...adding console servers isn't going to make you lazy. It WILL save you time, which you can spend doing some of the other tasks on your plate. It is well worth the investment of time and money to set it up.

-Zonker-

All Together Now...

2007-11-29T19:06:00.000-08:00

You use Console Servers to manage the serial ports of various devices, right? So, it only makes sense that a 'Best Practice' would be to connect the console ports of your Console Servers to a port on a console server, so you can monitor the Console Servers, too, right?

BUT, to do this 'right', you need to connect it to another, different console server. After all, when do you really need to use the console? When the machine is having a problem, and WHEN THE DEVICE REBOOTS! (All Caps used for emphasis...sorry if I offended anyone. ;-) When the device reboots, your serial console is how you see what the machine is doing until it gets going sufficiently for the network to be of use to you. So, if you connect your serial port to another port on the same Console server, you can monitor it when all is going well, but when things go badly, you've just lost your console access as well.

Tonight, I hit a bit of a snag...because I had connected some Console Server console ports to other "nearby" Console Servers. (Imagine 18" console jumper cables...why not, the 'different' units were close enough.) "What could be the problem with that", I had thought.

Tonight was the night to upgrade the software on this set of Console Servers...complete with systematic reboots between revision levels. Doing the three in parallel, to take advantage of 'cut-and-paste' to minimize the chances for typographical errors. Halfway through the reboots, I stop seeing progress...because the reboot of Console Server A had taken out the console access for Console Server B...

So, I type this note as I wait for the Console Servers to reboot and for my console access to stabilize again, and I consider whether my tie would have been worth the extra ~125 feet of cable to make the console runs to diversely-disparate Console Servers, to minimize the chance of a 'cluster service' outage taking out closely-grouped Console Servers. Food for thought.

-Z-

Whether we need DCD more than symetry

2007-10-23T19:02:00.000-07:00

I'm very much in favor of a symmetrical RJ45 wiring schema for console servers, because it could minimize how many adapters I would need to stock. The advantages of a symmetrical pinout include using a simple 'rolled' cable to make a 'null-modem' connection between devices and adapters.

Now, I'm not picking any particular vendors wiring scheme as 'the best'. I'm espousing symmetry as the 'best way'. The problem is, this means having two Signal Ground pins, which usually means that DCD won't have a pin. (On an 8-wire RJ45 interface, you would have two ground leads, two data leads, two hardware flow control leads, and two hardware handshaking leads.)

Remember that today's Console Server has its roots in yesteryears Terminal Servers. The old TS used modems or terminals, to let remote users connect to hosts across the network. The hardware handshaking was used by the terminals to indicate if they were connected. With modems, they could be connected and powered, but it was DCD signals from the modem to the TS which told the TS whether an interface was 'on-line' with a remote user.

But, these days in console service, modems are a pretty rare thing. Still, I'll be the first to recognize that modems could be used to dial into a remote site and try to connect when the main WAN links are down. However, you can configure most modems to toggle the DSR lead to reflect the status of DCD, and you can configure most console servers to recognize the DSR lead as a signal that the device isn't ready. (So, I don't think we really need DCD in console service. I'll leave Dial-up Networking for later article.)

If you look at my Signals Page, in the RJ45 section, you'll notice that there are already a couple of symmetrical pinout schemes. But you can also see the wide variety of combinations that I've found so far. While pin 4 seems to be the most common pin for signal ground, its clear that not everyone uses pin for for the signal ground. That makes it harder to make an RJ45 signal tracer that works in all cases, so there is less incentive for a vendor to take on that project.

So, for the existing symmetrical RJ45 pinouts on my signals page, you'll notice that they use pins 4 and 5 as ground pins. Data pins are on pins 3 and 6. Handshaking is on pins 2 and 7, and flow control are on pins 1 and 8.

If you're asking "why can't we use pin 5 for DCD?", then you may not have thought about the issue deeply. The first problem comes when you use a rolled cable as a null modem, and you now connect two devices signal leads, but not their signal ground pins! You'll be replying on the chassis ground connections to provide the signal reference. Using these console servers in large data centers means that you may be connecting devices on different phases of the AC circuit, or even across two different Power Distrubtion Units (PDUs) or Uninterruptible Power Supplies (UPSs). The result could be burned out serial ports on your console server and the attached devices, or more sever damage to the units.

So, to summarize the vendors I know about today;

Symmetrical, with 2 grounds (9+ of 26):
Cisco, Logical Solutions SCS, Lantronics SCS-1600, Digi PortServer II, Xyplex 1600/1800, iTouch/MRV in-Reach lines, Juniper Networks (J-series devices, and many NetScreen devices), Server Technology CDU consoles, and Sun Microsystems console/TTY ports

Symmetrical, except for 1 ground and DCD (3 of 26):
Digi CM, Digi STS-1610, Opengear CM

Of the 27 console connections I've documented, 15 use pin 4 for ground, and 9 use pin 6 for ground. (Keep that in mind if you are making your own RJ45 signal tracers.) And of the 9 symmetrical pinouts, eight of them use pins 4 and 5 for ground, and one uses pin6 3 and 6.

It seems to me that other vendors (who are not making console servers) have been choosing the Cisco wiring scheme (Juniper, Sun, Server Technology). Lantronix picked the null-modem compliment to the Cisco pinout. Those choices have been good for me, since it simplifies some of the deployments that I work on. It's interesting to note that the Xyplex/iTouch/MRV pinouts are almost identical, except that pins 1 and 8 (the flow control leads) are inverted from the Cisco schema. (So, if you don't care about flow control, you can mix-and-match these devices with the Cisco-schema cousins.)

I'm under the impression that some vendors are considering adopting a different wiring scheme for their future product series. I'd strongly urge that they consider adopting a symmetrical schema, and I'd also strongly encourage them to consider using an existing schema, rather than trying to come up with a unique schema. And I'm always interested in discussing it in person, though an email on the topic is also welcome.

Sometimes you want it raw...

2007-09-20T22:31:00.000-07:00

Your console connection, that is!

When you are using a console server for remote access to serial consoles, you're defaults are usually a telnet-type listerner on the console server. That means a 7-bit session, with the high-order bit normally being "0". (That's why you sometimes have a session that displays unusual characters...something trips the high-order bit to "1", and you need to reset the session/port to clear the bit.)

But, what if you NEED to pass 8-bit data? Sam Crooks recently had the need to upgrade the OS on a router, but didn't have a the ability to use the network to get the code to the unit. He could, however, use XMODEM-1K from a locally-attached laptop serial port to the console of the unit. Except that his laptop was across the network. The gotcha here was the 8th-bit was set to "1" in much of the data, and certainly in the file transfer sequence numbers. This botches up the connection, and you need to start over again. (I'll explain about the modes first, but I'll also share notes from Sam's solution towards the end of this article.)

Fortunately, most console servers today have legacy roots in "Terminal Server" service, as in dial-in modem pools, and binary file transfers. So, besides their 'default' 7-bit session mode, most have an "8-bit, escapable" mode, and some even have a full "raw 8-bit, non-escapable" mode. You just may have to dig around a bit to find out how to access them. To demonstrate the issues, we'll use Cisco access servers as the example.

With the Cisco devices, to reach across the network to a particular serial port, you normally use a telnet client, to the IP address of the access server, at the TCP port 2000 + n, where n equals the (async) line number. That is, line 21, would be at TCP 2021. This is for the normal, 7-bit session. So far, so good?

If you need to use the 8th bit regularly (specialty keys, alternate characters, etc.), you can get the "escapable" 8-bit mode by using the TCP port 4000 + n. (The same n as before.) The 'escapable' pat means that you still have a couple of the 254 possible characters that you cannot use as data, because these will be used for the client to escape from sending data, so it can send control info (like "quit this session"). Most folks could use the 7-bit mode or the 8-bit mode interchangeably.

But, if you need to send ALL 256 possible characters (think the block sequence numbers, if you think that your data wouldn't have all the characters), then you need to use a "raw 8-bit" mode. But, if you can't escape from the data session (because your escape characters are "just data" now...), how do you close the session? You need to get the device connected to the async port to toggle it's handshake line. (Think of it as a modem call ending, and the modem drops the DCD line, or it cycles the DSR line...because that where that code has it's roots.)

In the Cisco access servers, you can invoke the "raw 8-bit" mode at TCP port 6000 + n.

Here are the notes from Sam's solution, once he discovered these different modes;

The serial speed of 9,600 bps is too slow for big file transfers, so he increases the port speeds to 115,200 bps. (In his case, he uses autobaud on one side, and manually sets the speed on the other. I presume this is so the speeds will resort to 9,600 after everything's finished.)

Next, he sets the port for the data transfer;

Switch#terminal download

Switch#

Switch#copy xmodem: flash:/c3560-advipservicesk9-mz.122-37.SE1.bin

Destination filename [c3560-advipservicesk9-mz.122-37.SE1.bin]?

Begin the Xmodem or Xmodem-1K transfer now...

CCCCCCCCC

now here, we go to TermTerm's, menus....
File../Transfer->/XMODEM->Send...

Pick the file, select 1K at the bottom of the window (vs CRC or checksum)...

and off the file transfer goes....hopefully to completion, and in a reasonable time...

Finally, when the transfer completes, he needs to break the session from the far end, so he uses the command clear line n (where n is the line number of the serial port you are using for the transfer).

Cisco also has a good, older page about the various commands related to configuring access servers for modem usage, which has some useful clues as well, including transport in and out, and changing your escape characters. You'll find references to these binary modes on this page, in the section relating to 'rotary'.

You can also find useful information searching for "configuring lines for modem support". For example, you can find a brief summary related to contacting the AUX port on a Cisco DSLAM using these three modes here.

Related Cisco commands (Anyone can search the public Documentation part of the Cisco website, but Cisco support contract holders can find more information if they are logged in.)

reverse telnet
configuring modem line rotary
telnet break-on-ip
exec (look for telnet-related BREAK info)

Good luck with that!

-Z-

Insert Wire A into Slot B (specialty cables and adapters)

2007-09-19T14:10:00.000-07:00

The other title for this article could be "Pocket Notes (you really CAN take it with you)". Here's how I 'remember' how to wire a serial crossover cable when I'm in a data center, without easy web access.

I used to carry a small pocket notebook with notes and adapter diagrams, but those got worn and ratty and dog-eared. So I kept them in a computer, printed the notes and drawings, and carried a small (5" x 8.5") binder. This way, I could mark-up the notes when I was away from the computer, and make changes on the computer, and print more 'clean' pages. It was also easy to give a page away, since I knew I could easily replace it. Eventually, the binder got bigger, thicker, and heavier, until I finally started carrying the computer around.

Now, ten years later, I keep my console information pages on the web, but I also keep critical notes on my Treo (Palm OS device). But the OS isn't important! If you have a pocket/palmtop computer, you can use the memo functions to carry useful cable info in the note files.

ASCII art isn't easy, and it isn't pretty unless you are using a mono-spaced font. But recording the wiring for specialty cables is easy. In an earlier posting, I described how to determine the wiring for specialty cables. And most of us could easily spout off the wiring sequence for an AT&T 468B-wired CAT-5 cable, right? So, you start with that color sequence on one end...but what colors do you need on the other end? THIS is easily described with text, and proportional fonts do not get in the way.

I usually show the cable, wired from both ends, since I can use a simple RJ-45 cable continuity tester...but, I add the Cat-5 color code in between the pin numbers. I use the AT&T 468B sequence on the first end, since it's a good visual starting place. Then, I show the sequence on the 'other' end, with the resulting color sequence. It's this other sequence that it hard to translate in my mind, so I keep a 'note' that won't get dog-eared in my phone, since it's almost always going to be with me.

Here's an example of a crossover cable between Cyclades ACS Console Servers;

Cyclades crossover
one end...
1 whi-orn 5
2 orn-whi 8
3 whi-grn 6
4 blu-whi 4
5 whi-blu 1
6 grn-whi 3
7 whi-brn 7
8 brn-whi 2

other end...
1 whi-blu 5
2 brn-whi 8
3 grn-whi 6
4 blu-whi 4
5 whi-orn 1
6 whi-grn 3
7 whi-brn 7
8 orn-whi 2

Using the PalmOS memopad feature, I have note files for a variety of console servers. It's easy to combine wiring notes (crossover, loopback, signal inputs/outputs) for a single vendor into a separate file, for quick reference. You should be able to do something similar with other PalmTop devices.

I do still keep post-its handy at a few sites, with the "other end" wiring sequence, in case the memos aren't available. (The battery died, or I can't take a camera-phone into certain data centers...) When they start to get worn, I make new ones, using the records on my phone, or my web pages.

I hope this note is helpful to some of you. I know that it saves me a LOT of time.

When did my console service become Critical Infrastructure?

2007-08-31T00:39:00.000-07:00

Many Conserver instances were started as an experiment. It was something to add to an existing console server deployment. It's usually installed on a seldom-used machine, probably older hardware, and maybe without support. But, it was certainly "enough to do the job, for now."

Once it was working, a few other folks started to use it. Then you offered to add more ports into the config for other system and network administrators. Soon, you've pulled in most of the original console server ports in the shop, and you're buying more console servers, and you're starting to look for more RAM and bigger hard drives. You're wondering if you are backing this system up, since the logs have become useful data. Then, one day, some data retention policy comes along, and you realize your console data just became Vital Records, and needs to be protected. It's time to think about an upgrade, and a service contract, and a way to write directly to archival media.

Why didn't you think about those things sooner? That's the topic for today's blog. What started as a demonstration just became critical infrastructure at one site I support. Here's what we are considering as we are looking for making this Conserver deployment a "Production Service".

Supported Hardware. Rather than a hand-me-down or a "Frankenstein machine", consider getting newer hardware. This will give you a longer product life (read that as "you can get tech support and replacement parts from the vendor" for 3-4 years), and you should consider the support contract, since this will likely become Critical Infrastructure if it hasn't already.

If you are trying to do this "on the cheap", you could try using a hand-me-down machine. Make sure you get a spare chassis (with power supply and motherboard), and and many drives and RAM! Remember, older drives and RAM get to be more expensive when they are no longer the new stuff! You'll also need to be able to service your own gear, on your own time.

Redundant Power Supplies. Unless your data center has fancy power distribution units that source two circuits to a since power cable, you should consider using a chassis that has dual power supplies. Make sure that the chassis can run fine (fully configured) on just one power supply! You should make sure that you are sourcing the power supplies from two different circuits. Also, find out if the power supplies have to be on the same PHASE of power, and find out BEFORE you plug them in. (Have I mentioned the value of a support contract for your hardware?)

I need RAM. Lots and lots of RAM. But how much is "lots". This depends on the number of consoles you plan to support, including your "someday" scenarios. Remember that Conserver starts with it's own process, and then spawns children for (generally) every 16 ports. Your OS will want some. And any other tools and scripts will need some. (If you are going to be editing large log files, searching large data sets, or processing many large log files, you're going to want a LOT of RAM, so that you avoid swapping memory to the hard drives.) Don't skimp on RAM.

Dedicated Log Data Drive(s). You don't want your logs on the same drive a your main OS. (If your console logs fill the disk over a holiday weekend, and your system can't write it's own system logs, your Monday morning is going to be a LONG morning!) How much space do you need? This is the hardest thing to estimate, since it really depends on how much you use your consoles. But, here is a good ballpark to start - estimate 20 MBytes for every console you plan to support, plus 1-2 GBytes as a buffer for when some logs start filling up faster than you expected.

Since Conserver can auto-rotate the logfiles when they reach a certain size, you want to consider how large a file you can open with the editing tools you want to use. I rotate my logs between 10-20 MB, and I use grep and PERL to find things. But if you use other tools that can't open files larger than, say, 5 MB, then you should adjust your log rotation size accordingly.

Use RAID to help protect your data. Even basic mirroring of your log drive will help protect the data set (your vital log files). If you are capturing a lot of very busy log files, you might want to consider striping the data. You may also want to consider hardware RAID support. (This can save you system some CPU overhead, but it also adds some hardware complication. Still, many sites are successfully using hardware RAID. If this is your first attempt to use hardware RAID, you should consider getting the hardware support contract for your machine.)

Consider using a RAID pair to protect your OS, scripts and tools as well! Many newer machines can host five 2.5-inch drives in a 1-Rack-Unit chassis. That gives you a pair for the OS, and a pair for log data.

Backing up your critical log files to archival media. CD media is cheap, but so is DVD media. (Heck, DVD drives are cheap, too!) You can get 5+ GBytes on a single-layer DVD, and must laptops and other workstations can read them. It's an ideal media for the day when the auditor comes and asks you to produce the log files for certain machines across a range of dates.

Remember, too, that most log data is going to be ASCII data. It's VERY compressible, and you can use PERL and cron jobs to compress the latest newly-rotated log files to a gzip version. This will let you store more log data longer, and lets you delete some of the older non-zipped versions. (Compressing the log files means you are going to be backing up the compressed logs, so you can store more of them on the DVD media...this means some savings in the number of discs you need to write over time.)

So, what would I recommend? Let me start by giving you the information that I'm using to base the decision, and then I'll tell you.

Given:
650 console ports today, could grow to 1024
(20 MB x 1,024 = 20 GB of drive, plus overhead, 25 GB minimum)
I want to store large amounts of compressed log files.
(3 yr retention needs for some files, 60 GB minimum)
1024 consoles means 65 processes, minimum, but more if my processes are really busy due to verbose logging. Could be 128+ Conserver processes later.
I want to run Splunk for log checking (RAM and drive implications.)
I also need drive space for backup, and log report manipulation.

What I'm proposing;
Dell 2950, with two 1.8 GHz Quad-core processors (Energy Smart)
Pick your OS (I'll pick Suse Linux, 3 yr license for the OS support)
8 GB of RAM (four 2 GB Dual Ranked DIMMs, Energy Smart)
Hardware RAID
76 GB, 15k-RPM drive threesome for the OS
146 GB, 10k-RPM drive threesome for the logs
Dual PS

This gives me a pair of drives for the OS, plus an on-site warm-spare, plus a pair for the logs with a warm spare. The drives are hot-swappable, so if I have a failed drive that RAID cannot recover, I'll yank the failed drive and swap in the warm spare, then let RAID rebuild it, and then I'll call Dell support for a next-day replacement, for the next three years. If I were really worried about hardware failure, I might opt for the 4-hour on-site contract.

Total cost comes in at $8.5k(US) today, from Dell, though the price may be better through other channels. (That works out to about $2,900 per year, or about $240 per month, to support 1,000 vital ports in the shop.)

Can you get by for less? Certainly. But, what is the cost to you if you lose log files you need to be retaining? What will the business impact be if the server is down for a day or more, and your administrators have to scramble to manage their servers and network gear 'the old fashioned way'? The cost isn't too bad, for Critical Infrastructure, now that it's proven itself to be useful and reliable. Shouldn't you put it on reliable hardware as well?

-Z-

Making progress, leaving footprints.

2007-07-14T12:19:00.000-07:00

It's a bit depressing to see that I'm down to about one blog per month. At least I'm making a bit of effort hre, as well as making progress on the other projects that compete for my time.

As I've changed jobs, I appreciate the 'Oral History Project' that most companies use to store and share knowledge about network infrastructure, legacy reasons for doing things, even network architecture. But, I only like them a little, mainly because they aren't a stable source. People leave companies, or even just transfer to other departments. Stories change with time, and details tend to degrade very rapidly. I like documents better. (That's one large reason I make my web pages, and keep this blog.)

I learned a long time ago, from two examples, that the time it takes to make some basic documentation is largely paid ack when you use the documentation 3-4 times. Whether it's time saved when you refer to it yourself at a later date, or whether it's time saved when you give the document to someone else, and they can do a task, without you needing to give them the information via storytelling and sitting with them while they do the task. (I won't put the example stories in this article, to save space. Maybe I'll post them in a separate article, if there is some demand for them.)

When I'm debugging a problem, I keep notes, usually in a 'notepad' or 'teach text' file. (Save the file with a useful name, and save it often during the project!) If I'm working on a device configuration, I keep a diary of the changes I made, in case I need to revert anything. I also paste in any error messages I get during the process.

BUT, here's the message for this blog... During the process, I make notes about some of the useful options for some of the commands I'm using. (I needed to look through the syntax for this effort, and I learned something...so I save it for a future effort!) When I'm done, and I've figured out the proper sequence of commands, I now have a basic document that shows me the correct steps, and a few added notes for related commands. If I take an extra 30-60 mintes, I can add some comments for the next time... things to be aware of, pitfalls to avoid, information to gather before you start, how long it might take... these all get into the document.

At the end of the effort, I now save a copy of the file in a "Clues" directory. This is my 'canonical' place for the most current version of my clues, for anything I've worked on. I've written my stories down, and everyone on the team knows where to look for them if they need to work on a project. If I'm unavailable, my team will still have access to useful knowledge on an array of tasks. (And, if it's easy to hand off a project, it's more likely I'll be able to hand it off and do something more interesting the next time.)

Make a clues repository, start gathering simple process docs, and get the other folks in your department to share in filling the repository.

-Z-

The Emergency Kit

2007-06-02T13:31:00.000-07:00

I've put together a small kit, which can give me quick access to the serial consoles of various machines, for those little emergency trips to the colocation facility, or a customer site. It's been so successful and handy that colleagues are borrowing it for small cluster jumpstarts, and even for a Disaster Recovery exercise. So, I'd also describe it here.

The gist is, I started with a small 8-port console server, so I could use my laptop, a pre-configured telnet client, a crossover-Ethernet cable, and a handful of adapters to connect to various devices. I found that I often needed to be connected to the machine's network as well, so I added a small hub to the kit, as well as a small power outlet strip to my laptop bag.

I picked the Cyclades TS-800 8-port console server because it was small, was BREAK-safe, and ran from a 'wall wart' (power pack). I really wanted one that could run on 12-volts DC, so I could run the kit on car battery power, but couldn't find one. My laptop can also run from 12-volts DC (which could be handy in an emergency). Sadly, the TS-800 units are no longer sold by Avocent, but there are still a variety of small console servers on the market.

The small console server fit in the large pocket of my laptop bag, but limited how many cables I could carry, along with all the adapters. I finally fell to using a small, extra bag for the console bits.

I added a small 10-Mbps hub initially, so I could also do some packet sniffing, since most small switches don't have the ability to 'span' or 'mirror' ports. But, that bottleneck also throttled network access to the attached device, and I wasn't doing so much packet sniffing, so I 'traded up' to a Allied Telesyn AT-FS708LE 8-port 10/100 Mbps switch. This is a 12-volt DC device. It's small, lightweight, sturdy, and still available cheaply.

The kit now includes the switch, the console server, two crossover-Ethernet cables (1'- and 5'-long), an assortment of lengths of regular 8-wire CAT-5 cables, and an assortment of 24 Cyclades serial adapters. I also have a few 1'-long Cyclades null-modem (DCE-to-DTE crossed RJ-45 cables, and a few CAT-5e-rated female RF45 couplers. Add in the wall warts, and the kit lives in a 6-quart Sterilite plastic container about the size of a shoebox. Easy for security folks to see what is inside, but you don't want to put this in the belly of an airplane.

The Console Server is really the key to this kit, so here's what I've done to make it work;

First, configured the device as a console server, then copied the config file so I have an on-board copy, in case someone feels the need to change settings while they borrow it.
cp /etc/portslave/pslave.conf /etc/portslave/pslave.bkup

Make sure the backup config is saved into the unit's NVRAM. On the Cyclades, this means adding the file into /etc/config_files. (This is one reason I strip most of the comments out of my console server config files, since I'm basically saving two copies of the configuration file.) Now, if someone needs to change the config file, it's quick and easy to get back on the console, and copy the config back over the pslave.conf file, then 'saveconf' and reboot.

I also add an entry for my laptop into the /etc/hosts file on the console server, to reduce delays when making your reverse-TCP connections to the console server.

I had originally assigned different port speeds (19.2k, 38.8, 57.6, and 115.2) to ports 5 through 8, to make it quick and easy to access a high-speed port. But, I found that most ports are STILL at 9.6kbps as a default, and sometimes I needed more than 4 ports. In those cases when I needed a higher speed, I often needed more than one port at the higher speed at the same time. Since I found myself making many configuration changes I decided to revert the base configuration to 9.6k all 'round, but that's just explaining my reasons. You may want to do it your own way, and that's up to you.

I also pointed all the "extra" services (Radius server, syslog, etc) that the console server might want to use to the laptop IP address. I already use a small but of RFC-1918 address space, to reduce the chances of a conflict with the network that I may attach to. Pointing just to my laptop reduces the chance of the TS tripping any Intrusion Detection System (IDS) that might be watching the network. (I don't use the Syslog much anymore, but if I want to try it, that's one less part I need to configure.)

The current laptop is a PC, using PuTTY as the client tool. This lets me save the basic settings for "ts-1" through "ts-8" (configured to connect to the IP address of the console server, at ports 7001-7008, respectively). I use a distinctive window size and text/background colors for these windows, and each connection is configured with a unique 'window name', and a unique log-file name. The distinctive window settings make it easy to tell which open windows are serial connections, versus network sessions when I'm debugging.

An advantage to using PuTTY for this, is I can also 'export' a copy the Windows Registry settings for these sessions onto my FLASH fob, and easily give the settings to someone who is going to borrow the kit, and VIOLA! They are ready to connect to the ports quickly and easily. PuTTY is small, and runs without needing to be 'installed', so I have a copy of that on the FLASH fob as well.

The final bit that makes the kit as easy to loan as it is for me to use it, is a 1-page README document (also on the FLASH fob...with a canned Conserver config snippet for this device, just in case), which outlines the default IP settings, serial speed settings, adapter and cabling clues, and basics about how to open a reverse-TCP session to a serial port. I also include clues and the password for configuring the device. (Add a few useful labels to the hub and console server, mark the wall-wart connectors with a color-code at the device-end, and things go pretty smoothly.)

I don't use it very often, but when I need it, it's ready to go! When your time is worth money, having a kit on the shelf is money well spent!

-Z-

Relieve pressure in your shop!

2007-04-22T15:11:00.000-07:00

We've got plenty of pressure with deadlines looming, new projects, and other 'people issues'. But the pressure I'm writing about today is torque.

For serial adapters that screw on (DB25, DE9, etc.), you shouldn't need to tighten them more than finger-tight. The screws are meant to keep the connector from being pulled out if the cable is tugged. Most of the small threaded nuts and screw-end on the adapters are not made for serious torque, and will likely break, or strip, if you apply too much pressure.

Finger-tight for you may be tighter than finger-tight for someone else that might need to work on the gear later. I don't want to make someone have to go get tools to remove serial adapters.

This goes for the screws for mounting your gear in cabinets or relay racks. You can tighten these hand-tight, but don't set your screw-gun to be an impact wrench! The lowest settings will probably be fine. (If it's not, you should make sure that you haven't cross-threaded the screw to begin with, rather than cutting new threads in the rails...because aluminum rails will strip easily, and steel rails will chew your screws easily. Let's try to not damage any parts today.)

The other damage that occurs from too much pressure is damaging the screwdriver bits and the screw-tops. It starts when the screw stops, and the drill keeps going, and the screwdriver bit starts to round over the screw head, but it's starting to damage the screwdriver bit as well! This makes it easier for the screwdriver bit to round over the next screwheads. (Take a good look at the philips bit in your screwgun today. If you don't have good, straight edges, it's time to turn down the torque, and either get out the file to renew your bit, or use a new replacement bit.)

The pressure you will save, will be the pressure on you in your next emergency. You'll be able to quickly remove an adapter without needing to go get tools. Your tools will easily remove and install screws, and your screws and rack rails won't be damaged by 'the last guy that worked on them'.

Once you set the torque on your screwgun to only 1 or 2, keep checking it, since there are probably other folks that borrow it. They'll likely change the setting, and not put it back. When you find it has been changed, you may be able to share this note with the culprit, and make things just a bit nicer in your shop.

Use the correct serial adapters, and trust them.

2007-04-22T14:51:00.000-07:00

Today was a service window, and I was working in an old part of the network. I stumbled onto a broken DE9-toRJ45 adapter, dangling by it's wires. I swapped it out for a new one, from the proper vendor for the terminal servers in use, but there was no response via conserver.

"It was working fine last week." I'm told.

Looking further, I found a poor RJ45 termination, but the signals on on my signal tracer looked odd as well. This led me to the other end of the cable, where I found that I was looking at a USOC-Rolled cable (old CDDI-type wiring). This would have been a 'null-modem' if they had been using Cisco console servers, but they were not. Instead, they had only a couple leads correctly connected, and signal ground was not one of them. After a bit of frustration reaching more consoles, and reterminating those cables to be straight-through again, the network was back under our control. But why had they failed?

"But they've always worked like that." I'm told.

The wiring of the Cisco adapters, the null-modem (USOC-Rolled) cables, and the non-Cisco console servers wiring scheme had meant that signal reference was really be derived from the equipment chassis, across the power systems ground, between the console servers and the controlled devices. Today was they day that newer, fatter power circuits were installed, and moving the gear to different Power Distribution Units were too much.

The signal reference ground pin is extremely important, to prevent data errors, possible equipment damage, and possible injury to staffl handling the connections! (I'll share my 'bad ground, smoking gun' story another day, when I have more time.)

Using the correct adapters, and correct wiring, will lead to predictable results. Mixing and matching parts you have laying around with specialty cables will eventually cause you more headaches, and possibly more money than you will save. And the chances are good that you'll need to 'do it the right way' sooner or later. If your shop is as busy as the places I've been, you may as well do it right in the first place, because you may not have time to do it right later without sacrificing weekends or vacation time.

Kudos to the Cyclades folks

2007-04-13T22:46:00.000-07:00

Their product was developed across three continents, and sometimes you could find grammatical errors in their on-line tech notes (perhaps because the authors first language was not English?), but I have not found any techical errors in the command-line instructions of the older files.

To be honest, I occassionally took some liberties, trying to optimize the steps that they had outlined. But, if things didn't work the way I expected, I have always had success when I did it exactly the way they typed it.

As someone who owns and operates on a bunch of older Cyclades gear, I'm very thankful that Avocent has retained the old Cyclades files at http://global.cyclades.com. Access to the old information has proven to be invaluable for upgrading older units to newer code.

And, as a fan of documentation (see my Console Connection Guides...), I tip my hats to those who made sure that the original instructions were spot-on correct, every time! Thanks, folks! I hope the Avocent team will follow well in your footsteps.

-Z-

Going back to the beginning

2007-04-08T21:42:00.000-07:00

"I am waiting for you, Vizzni! You told me to go back to the beginning, so I have. When a job went wrong, you went back to the beginning!" - Inigo Montoya, Princess Bride -

Today's topic is a lesson in Change Control, sort of. More like Pre-Change Control.

Most manufacturers post their upgrade instructions. Some of these instructions are fine. Some are even exceptional. But few manufacturers offer instructions for how to downgrade their code. Yet, if an upgrade goes wrong, you will probably want a fall-back plan, to get you back where you were, at least until you can get the vendor to help past the problem(s).

Most console server devices store their OS and boot files in FLASH. But many of them cannot store more than one version of the OS in the on-board FLASH at a time. So, the upgrade process will overwrite the old version. If you plan to revert to the original version, you need to gather a few things before you start;

A copy of the current code version
(You may be able to TFTP the file to a server from the device!)
At least a byte-count for the installed currently installed version
Better is to have the MD5 checksum hash for the currently installed version

A timestamp for the currently installed version may not be very useful, because many devices display the current system time for the files in FLASH. Even if the device displays an older timestamp, it's often the date the code was installed, and not the datestamp of the file was created. Still, if your device gives you older timestamps, it's worth noting the date and times of all of the files in your FLASH, before you start, so you can compare them at the end of the process.

If your device has MD5 implemented, so that you can check the new version for integrity, you should also use MD5 to display the checksum for the current version before you start. (If you need to load the old version back again, you should know what the checksum will be.)

You keep all of the software CDs that came with your devices when they are new, right? (At least ONE copy for each platform, and for each different software version as you collect them.) You should also mark them (the CDs, Manuals, etc.) for the software version they contain. The cost of a bookcase is cheaper then an evening of your wasted time because you cannot easily find the information you need.

"But my vendor keeps all of this stuff on-line. Why bother to keep the old stuff?" you ask. Let me offer a few cases that happen more often than you might expect;

The vendor goes out of business, and the website goes away.
The vendor sells the product line, and the new vendor kills it off.
The vendor gets acquired, and the old website goes away.
The vendor decides old software is "support", and is worth money.
Newer versions of software are too big to fit into the FLASH in your device.
The upgrade instructions had a mistake.
The new code doesn't like your old hardware.
The vendor decides to remove old versions from the web, to encourage the adoption of newer versions.

You should make a habit of checking for newer coder versions, for all of the hardware devices you own. Read through the release notes, and decide if the code might be useful to you. If you might want to upgrade someday, collect the newer versions (the binary file(s), README file(s), and PDF documents, and any checksum files).

Keep the versions in separate directories, since sometimes the image names are identical, and you don't want to overwrite the old files when you add the newer files. Then, you will be ready to upgrade when the time comes, and you'll also be ready to downgrade again, if needed.

-Z-

Serial Speed-shifting demons

2007-03-31T11:33:00.000-07:00

A fellow called Jerry Roy emailed me, asking if I knew how to connect a Cisco AUX port to a Topglobal Phoebus EVDO router console. There were a few thing he needed to work out...

1) the Cisco RJ45 to EVDO Mini-DIN-8 cable wiring
2) the configuration settings on the Cisco
3) why his old Cisco config clues didn't work right

Jerry is no slouch at this stuff, and had configured Cisco gear for reverse-TCP conection to the AUX port in the past, but this one was baffling him. He had a mini-DIN to DE-9 modem cable, but it wasn't playing nicely with the Cisco adapter. When he connected to the AUX port over the network, he got a Cisco login instead of the EVDO. Solving this took us about a week, including emailing to the vendor's support staff and the EVDO manufacturer in China to clarify signalling info and determine what wires needed connect to which pins. Once we had what we thought were the right pins, we still saw garbage on start-up, which made us try a variety of settings (even though the console speed was set to 9600 8-N-1).

We determined that the exec/no exec flag had changed in a major Cisco IOS revision from 12.2 to 12.3, so his abbreviated configuration commands needed to be amended. (It doesn't hurt to invoke a 'default' setting, so my Cisco config clues are a good start, if you're using reverse-TCP to get to Cisco ports.)

Jerry finally determined that the Tpoglobal Phoebus EVDO device runs its console at 115.2k 8-N-1, but he couldn't get any reponse from the console after the startup...because the OS then changes the EVDO console to the user-defined speed. The EVDO was running POST at one speed (spewing garbage at the modem/console), then silently shifting speed to a different, user-specifiable speed. He eventually set the user-defined speed for 115.2k, so the speeds would be consistent.

What made this so hard;
a) cisco changed the 'default' functions for local login (exec) between major IOS versions
b) vendor docs used unusual signal names (TD+, TD-, etc.) without input/output clues
c) molded cables, and unusual connectors made it hard to determine cable pinouts.
d) Roy didn't have any signal testing gear besides a voltmeter.
e) vendor docs didn't mention that P.O.S.T. messages would always be at 115.2 kbps

Developers Clue #1: If you think this speed shifting is a good idea, at least end the POST with a tag indicating that "POST is done, switching to OS console speed", even if you can't indicate what the new speed might be.

Developers Clue #2: If you think this speed shifting is a good idea, in the code that allows the user to set/check the definable console speed, you should also retun a note that the POST speed will remain at [speed], so the user is reminded that there will be a difference. (Or add an option to set the speed "Same as P.O.S.T. console speed".)

Tech Writers Clue: If the system console can be set for two different speeds, the cases need to be called out in the manual.

Here's the cable diagram Jerry came up with (in case someone else needs it) with the pin numbers shuffled to see where loops occur;

Cisco AUX           EVDO console
RJ45M              mini-DIN-8M   refer to my Signals page
1 RTS -->(nc) (nc) --> DCD 1     for connector pinout info
3 TD  -->------------>  RD 3
6 RD  <------------<--  TD 5 8 CTS <------------<-- RTS 6 4 GND ---+-----------> GND 4
       |    (nc) --> CTS 2
5 gnd ---'  (loop Cisco pins 4 to 5)
2 DTR -->-.   (nc) --> DSR 7
7 DSR <---' (loop Cisco pins 2 to 7)

Since the EVDO apparently doesn't have a DTR output, we looped the Cisco hardware handshake lines. (For use connecting to other consoles, I would usually pass these to the 'far side', so the far device could tell if the cable was plugged in AND the near device was turned on. After all, that's what these leads were designed to do.) If you care about hardware flow control (RTS/CTS), you could also connect Cisco pin 1 to EVDO pin 7.

-Z-

Mining for (serial) gold on Google

2007-03-31T10:59:00.000-07:00

When I'm using my gmail account, I find that I know keep an eye on the 'content-relavant' ads they post on the right side of my screen. If a link looks interesting, I'll 'open link in new tab', and look at them later. This way, I don't really interrupt the flow that I'm following now. Most of the links don't pan out, but sometimes I find gold, that really valuable site that I might not have found using keyword searches before. (I don't know why I don't find them at other times. I'll take lucky chances any day! :-)

Today's nugget was the Intro to Serial Communications page at TalTech.com. There is a lot in common with my Minor Scroll of Console Knowledge, but there are a few other tidbits there that I haven't put on my page. (I've already emailed the webmaster, asking if permission to link to their page. Netiquette first, of course.)

If you haven't tried gmail, you might want to, now that you don't need a referral. My acount was a test, to chat with a friend. It expanded to become an archive for console-related emails that I received on my other accounts, and the ads became more relavant. Now it's one of my main accounts. Thanks, Google!

-Z-

Crossing a VPN to get to your Console Ports

2007-03-29T23:00:00.000-07:00

You may not have tried this yet, but you may want to read the note anyway, so you'll be prepared for the day you will need to cross a VPN. As security gets tightened, and networks are segmented, the VPNs will be coming to a LAN near you.

I hadn't thought about the issue of "dropped links" for years. My modems would often time-out from inactivity. Sometimes ISDN connections would drop due to idle timers, because there weren't that many B-channels available in the pool. I understand why you get timed out, and I empathize with the implementers. But, sometimes you need to keep a link up, without thinking too hard, or remembering to type a key.

In the modem days, I'd start a PING across a PPP link. Sometimes I'd set my email to poll every 5 minutes, just short of the timer, in order to keep links active. But broadband access and SSH to hosts was ubiquitous enough that I'd forgotten about this inconvenience.

Today, I was bitten bye a VPN with idle-timeouts set uselessly low for working remotely on my consoles. While email, and web browsers don't care so much (since they are always initiating a new TCP connection), it was invisible at first. But, my SSH sessions would suddenly be frozen. with no indication of the problem. Meanwhile, my idle TCP connection at the far host was abandoned, and would need to be killed off by someone with administrative privileges.

The answer was the TCP-keepalive options available in my SSH client. Under Microsoft graphical OSs, I use PuTTY, and you can set the option there for sending keep-alive null packets every n seconds, and/or TCP keep-alives. This solves the problem when it needs solving (only when I've got an SSH session I care about), and its automatically done (no need to remember special hacks), with minimal load on the network link.

Now, I admit that I am hogging a resource (the VPN session), but I'm usually logged in for a good reason (such as wrenching on the network), so I'm doing it for a good cause.

If you are going to SSH into console servers (or even reverse-TCP), a dropped VPN connection will cause you a lot of grief, since the port you were connected to will be 'busy' until someone can kill off that process. If you are using SSH to log into a Console Management Application Server (such as Conserver), you will be leaving abandoned sessions on the application server host, but the console ports themselves will still be available to the next person who needs to get on them. If you run into this symptom, check for the keep-alive options in your telnet/SSH clients, and see if that doesn't fix your timeouts. Just remember to log out of those sessions when you are done, so the VPN access can continue to be shared.

-Z-

Making the adapter or cable you need

2007-03-24T10:42:00.000-07:00

I've got shelves, covered with tupperware tubs and drawers of adapters, sorted by wiring type and DCE/DTE, and labeled (both the adapters, and the tubs and drawers). I also have a variety of test gear handy to help me test serial ports and devices. But I know that I'm a bit unusual in this regard.

Most shops I've visited have one box or drawer where they collect 'stuff' that isn't 'Ethernet'. The problem is, many vendors do little to label their parts, other than to add a part number. (Sometimes, though, the part number is on a baggie around an unlabeled part. Once the baggie is lost, the part becomes useless as soon as you seperate it from its associated equipment.)

You can spend a lot of time trying various combinations of the parts at your disposal, testing at each step whether it shows signs of working, or you can set out to build the cable or adapter that you need.

To build the cable or adapter, you need to know:
1) are both devices using the same Electrical Signalling (i.e. RS232, TTL, RS488, etc.)
2) which pin/contact numbers are which on the connectors you need to use
3) which signals are on which pins, and which are inputs vs outputs

Your search for information should start with the equipment you need to connect. Some equipment will include pinouts and signal info on a label on the chassis itself. Next comes reading the manual, which should be on a bookshelf in your office if the equipment is still in use. Sometimes you can find the documentation on a CD-ROM that came with the equipment (you keep those, too, right?), or the vendors website, in their support section. Start with these if you can, since configuration information for the port may be needed later, and these documents will be specific to the software revision loaded on your equipment.

If you can't find the information resources from your vendor, your favorite search engine may be useful. I prefer Google, but they have their good and bad weeks when it comes to finding useful results. (There are occassional bouts with bogus search portals saturating the Google results. It's like a tide, really...when it gets really bad, they manage to clean it up, and results get really good again. No, I'm not trying to bite the hand that holds my new blog. ;-) I've had poor results with MSN, and Yahoo is hit-and-miss but they have too many ads for my tastes. The lesson here is to pick your favorite, but to try others if you are not successful in your searches.

You want to be sure that you only connect two devices if they have the same electrical signalling specifications. TTL signals are from 0v to +5v, while RS-232 signals are -12v to +12v. If you connect a TTL device to an RS232 device, you may not damage the RS232 port, but you will most likely 'let the smoke out' of the TTL interface chip (e.g. permanently damage it).

Now, it IS possible to connect a TTL device to an RS232 device, using an interface (electrical) level converter between them, but you cannot do it with a simple cable or adapter.

Once you know that your two interfaces are electrically compatible, just make a wiring diagram for your cable or adapter. List one connector on the left side of the page, and the other on the right side. Include which devices are being connected, as well as the connector type and gender, so you can identify this wiring diagram easily if you need it again in the future.

DCE and DTE were defined 'in the old days', but it was a 'draft specification'. It was an attempt to make things interchangeable. But it wasn't something which culd be enforced, so there are manufacturers that picked the 'less intuitive' option when designing their devices. As a result, they stack the odds against us if we are just guessing. So, when you find the pinout descriptions for the signals, you could 'bet' that TXD (Transmit Data) is an output, and RXD is an input. But, if you're using a less-popular/smaller manufacturers device, you may be disappointed. That's why I test the pinouts after I know where the Signal ('Reference') Ground pin is located. You should try to test your ports as well, whether you use a voltmeter, or a signal tester.

I usually list the pin numbers sequentially (with 1 at the top), and I also add the signal name, and the 'direction' of the signal. If the signal is an output, I use an arrow that points away from the number. An input will point at the number. The reference ground signal will not have an arrow. (You can see an example of this on my Signals page.) You can even use this format in simple text files or email, like so...

Cisco AUX         Cyclades ACS
RJ45-M              RJ45-M
1 RTS -->      <-- RTS 1  2 DTR -->      <-- DTR 2  3 TD  -->      <--  TD 3  4 GND ---      --- GND 4
5 gnd* <-      --> CTS 5
6 RD  <--      -->  RD 6
7 DSR <--      --> DCD 7
8 CTS <--      --> DSR 8

You can now start drawing the lines that connect 'complimentary' signals, but I find that all of the crossing lines are difficult to read. Instead, I draw a second version, where I draw the signal straight across from the left side to the right side, and I put the appropriate number in the 'right-hand column'. It makes it easier for my eyes to follow. It would look like this...

Cisco AUX         Cyclades ACS
RJ45-M              RJ45-M
1 RTS -->      --> CTS 5
2 DTR -->      --> DSR 8
3 TD  -->      -->  RD 6
4 GND ---+--   --- GND 4
5 gnd* <-'   nc -> DCD 7
6 RD  <--      <--  TD 3  7 DSR <--      <-- DTR 2
8 CTS <--      <-- RTS 1

This gives you a chance to count all of the pins on the right side. Is there one of each number? It also lets you see the 'odd' connections. In this case, the Cisco side needs to loop pin 4 to pin 5 (it's a switch in legacy products that tells the interface to be RS232. Without that input, the hardware doesn't use RS232 signalling.) Since the Cisco doesn't have a DCD output, the input on the Cyclades would not be connected. You can program the Cyclades to ignore the state of the DCD input. By diagraming the cable or adapter like this, it's a visual reminder that you need to check whether the device cares about an input which will NOT be connected.

I also find it's helpful to have a reference illustration for the pinouts of my connectors handy, when it's finally time to make the adapter or cable. You can find illustrations on the web, but you can find the cmmon connectors used for RS232 connections on my Signals page. Scroll to the buttom of the page for the DB and DE connectors. You can find a more verbose collection of basic RS232 clues and signal information on my Minor Scroll of Console Knowledge.

-Z-

Kick-off post

2007-03-21T10:44:00.000-07:00

I've put the project of adding a log to my Console pages (http://www.conserver.com/consoles/) for too long. This seems like a good way to get started with blogging without any more delays.

My plan is to post tips and ideas, and ring to light some of the unusual problems I find while helping set up secure, remote access to serial consoles. I hope the site will become a supplement to the guides and the testing I do on the regular site, and I welcome questions and comments on the topics that I post.

Regards. -Z-