CW@50: Sharky's greatest hits
Editor's note: For almost two decades, Computerworld’s Shark Tank has been there for you, five days a week. For many IT professionals, it’s a quick escape that often brings a smile, frequently of the rueful variety.
If you aren’t receiving it in your inbox, you can find it here, and in case you ever have any of that elusive commodity known as downtime, you can dive into the thousands of dispatches from “pilot fish” in the Sharkives. As part of Computerworld’s ongoing celebration of its 50th anniversary, we’ve done our own deep dive and surfaced with these gems.
Route... er, root causeComputerworld
Big engineering firm is developing a vehicle-detection system for futuristic highways — very Buck Rogers. Developers work night and day tuning communications between the on-the-street test site and the lab — very plain-vanilla IT. Beta goes in Friday. Works all weekend. Monday it stops communicating. Panic ensues. Bigwigs scream that the beta’s a failure. Corporate presidents fume.
All eyes turn to the developer, who finally figures out what’s wrong. “Turns out the system was working fine,” a pilot fish on the scene reports, “right up to the point when a car wiped out the communications cabinet on the side of the road.”
Next time, aim for the ferret’s apple
After Sharky related the tale of an enterprising pilot fish who used a radio-controlled toy tank to pull network cable above a dropped ceiling, he heard from other pilot fish who have come up with creative ways of solving that problem.
One fish used cat food to train her pet rat to scurry over ceiling panels, under floors and inside walls, using its teeth to pull a long string toward a tapping sound at the other end of the cable run. Once the rat emerges, the fish attaches the cable to the string and pulls it through.
“It took about 20 minutes a day for three months to train Rattie to negotiate the maze, avoid dead ends and travel toward tapping sounds,” the fish says.
A little less dedicated to the task is a ferret that belongs to another IT pilot fish. Same basic approach — the ferret drags a string, which will eventually pull cable — but with an apple slice instead of cat food for bait. “It stops every now and then to scope out the territory,” says the fish. But, he adds, “in the end, the ferret goes for the apple.”
But the prize has to go to the ill-starred team that tried using a high-powered hunting bow to shoot the string the length of the room.[ To comment on this story, visit Computerworld's Facebook page. ]
“Everything was going great,” says a pilot fish on the team, “until one day during business hours. The arrow glanced off a sprinkler pipe and was deflected down through the suspended ceiling tile, where it embedded itself into the top of a very shaken office worker’s desk!
“Needless to say,” says fish, “we went back to the old-fashioned way of lifting ceiling tiles to route cable.”
Where’d he go?
It’s 4 p.m. on a Friday afternoon when this small IT group starts its HR software upgrade, with conference-call help from an analyst provided by the vendor. It looks like things are going fine, so 10 minutes into the process the analyst explains he has to drop off the call briefly for a meeting with his supervisor.[ See more Computerworld 50th anniversary coverage ]
“At 4:30, his supervisor called me,” reports a pilot fish on the team. “He informed us that our analyst was ‘no longer with the company.’ I asked who would be working on our conversion. He said, ‘We will have to assign someone to that early next week.’ My response: ‘But the conversion is running now!’”
Supervisor promises to find someone immediately, but fish can do the time-zone math — the vendor’s staff has left for the weekend. “The conversion did error out, and the vendor didn’t assign anyone until Monday,” sighs fish. “We now, only half jokingly, ask every vendor’s analyst if there is any possibility his employment might be cut short for performance reasons.”
And how did you fix it, exactly?
For weeks after new servers are installed, this computer room can’t stay cool. “The plant engineer installed a red warning light so we could see if the temperature exceeded 78 degrees, and it was always on,” says a sysadmin pilot fish. “I pointed it out for weeks, expecting them to do something. Finally the light was out, and I thanked them for fixing the cooling problem. No problem, they answered, glad we finally got it fixed for you.”
But next day it’s hotter than ever — and the red light is still off. “It wasn’t fixed,” grumbles fish after investigating. “The light bulb had just burned out.”
Now THAT’S a generator test!
This rural county government doesn’t have a lot of money for spare equipment — which means an emergency generator for the data center isn’t high on the county board’s list of priorities, according to a pilot fish on the scene.
“The IT department finally convinced the board to approve a backup generator for the computer room,” says fish.
“But it came with conditions. Since power emergencies are rare, they only approved a portable generator that will be shared with the road crews for their emergencies also. And while the IT director wants the generator stored in the IT parking lot, instead the road crew stores the generator at the public works yard, about 10 minutes away from the IT office.”
The IT director keeps pushing for the generator to be right next to the computer room — but after nearly two years, it’s still sitting across town.
In time, it’s clear to the IT director that he has lost the board’s support and is on the way out. But he decides to go out proving a point.
Early one morning, before anyone has had so much as a first cup of coffee, the IT director calls the public works director and says the lights just went dark and the power company doesn’t know why. It may take hours to troubleshoot the problem, IT boss tells the public works boss, and we need the generator now.
“Public works employees scramble to find a truck with the right hitch to tow the generator, and discover it has a flat tire that needs changing,” fish says. “After 45 minutes, the generator still hasn’t shown up.”
That’s when the IT director pulls the plug on the fiber, to simulate the data center going down.
None of the servers actually crash, of course. But all the government workers spread across a dozen remote offices lose access to everything but their own PCs.
No email. No server applications. No shared drives or print services. No Facebook or Amazon, either. And there’s pretty much no one on the county payroll who doesn’t feel the pain.
Deciding he has made his point, the IT director finally orders the fiber plugged back in, and everything goes back to normal.
“The IT director? His forced retirement was moved up by two weeks, but he otherwise escaped with his skin intact,” says fish.
“And the generator? It still sits across town at the public works yard — but now they check the truck’s tires every Friday afternoon before they call it a day.”
This story, "CW@50: Sharky's greatest hits" was originally published by Computerworld.