Whaddya Mean, It’s Not Broken?

That was an interesting few minutes.

This morning, I saw that one of the hosts at work, which is not mine (belongs to HVAC for the control system for the building – tells them about the status of the systems in here, and lets them change things remotely like the temperature of various rooms and such) had gone red and then green again. Now, it was red again, but when I got down there nobody was around next to it. Panel was left open, and you could see the ethernet, serial and power cables where the terminal server used to sit. So I waited, and as soon as I saw it went green I ran downstairs. Since the management station only checks every ~15 minutes for a host’s existance, I was hoping that it came online recently and someone was still sitting next to the box. Sure enough, when I got down there, Nate (I think that was his name, I’m terrible with names – and if somehow you’re reading this and it’s not your name, I apologize) was staring at the blinkenlights and wondering what to do next. I said hello, we exchanged formalities, and I said, “You know, that thing’s not broken.” “Well, we can see it’s there, but we can’t talk to it.” “Yes, I know. But it’s working just fine.”

See, when our firewall work was done yesterday morning, one of the last comments about the changeover was that two hosts that had been outside the firewall, on our POP [Point-Of-Presence] switch, were moved to inside the firewall. But since I don’t control this particular box, and I don’t know who does, I didn’t know who to contact about the fact that it would now be unreachable from outside the firewall (which by nature it would have to be, for them to be able to connect to and talk to it). So I waited, and today figured that the box’s controller and/or owner realized it was not talking.

We came up to my office, and Nate called Marty in the control shop, told him the story, then handed the phone off to me. In the mean time, I’d already loaded the firewall config, added this host as an internal host in the list, and was ready to add the external host to the list as well, when he asked me who actually determined that it should be moved. I told him how it was moved, and when he said he was going to call that person I chimed in, “Well, if you give me the name or IP of the computer this thing talks to, I can have it setup in the firewall in less than five minutes.” So he did, and I said I’d call him back while I added the other host and a new policy. Couple minutes later I called him back, and he said it’s working fine now.

During all of this, I also got the story on what had happend thus far that day. They started in the morning by resetting the box, but that did nothing of course. Then they took the box out of the building, over to their office, and reprogrammed it entirely. When I caught Nate downstairs, they had just reinstalled it and were hoping that would do the trick – though they’d just realized it hadn’t before I got there. Fortunately I showed up before he unhooked it and ran away again :> Also got someone from Siemens to look at it, and that person was scratching their head about the problem (which as Nate said, “I hate when he does that… ’cause it pretty much means that nobody has a clue what’s wrong.” So if any of you are reading this, sorry for the mishap – but without knowing who to contact about the box, to say, “Hey, this is in our firewall now, give me answers to these questions and it’ll work just like before,” all I could do is wait for someone to show up to “fix” it, and tell them that there’s nothing to fix.

Leave a Reply