The hook tool
Ever drop a screw into a machine where you can't get to it with your hand? You could use a magnetized screwdriver to get it out, but what if it's wedged somewhere, or underneath a board? That's where the hook tool comes in.
Also handy for (and originally designed for) installing/removing tension springs in laptop computers, the hook tool replaced an array of dental picks that I had.
Good old air-in-a-can. Useful for blowing the dust out of PCI/AGP/ISA slots, drive connectors, ATX sockets, fans, heatsinks, optical drives, and the computer chassis as a whole. You could use a vacuum cleaner instead (and for extreme cases, you should start with a vacuum cleaner and finish with Dust Off), but Dust Off tends to get a more powerful blast into smaller places.
The pry tool
If you remember one thing in your career as a hardware technician, it should be this: never add or remove components while the power is connected. If you remember two things, remember this as well: never pry plastic parts with metal. If you do, you'll chip the hell out of the plastic, and whatever you are working on will be cosmetically ruined.
I keep two tools around to pry apart plastic pieces (like laptop computer cases that snap together): one is an old 30-pin SIMM memory module, and the other is a plastic pry tool kit. Despite using softer materials to pry, you still have to be gentle and think your way through a disassembly rather than use brute force.
Power supply tester
You could lug around a spare power supply for testing, but you can get a higher degree of accuracy and reliability from the Antec ATX12V power supply tester that I recently reviewed. Not only does it show if the PSU is good or bad, but it also shows which specific rail is failing. It's small, inexpensive, easy to use, and reliable.
No matter what operating system you use, when troubleshooting a computer the first thing that you should do is isolate the problem. Is it hardware or software at fault? The best way to make that determination is to use a known-good operating system installation. If you have one installed on a spare hard drive or partition, fine. If not, get Knoppix and put it on a CD. If Knoppix boots and runs perfectly, you can be more confident that your computer trouble is software-related. If there are big problems with Knoppix (and you know the hardware is supported, which it almost always is), it's a safe bet that you have faulty hardware.
Every electronic computer part you buy comes enclosed in an anti-static bag. Some are clear plastic, others are made of a silvery substance called Mylar. Either will do for storage and transport of electronic components. You could buy some, but if you've purchased a lot of computer parts over the years, your in-house stock of anti-static bags should be sufficient.
Why do you need these? Because if you have to transport a computer part from one place to another, the only way to ensure that it will not be electrically damaged during its journey is to protect it with anti-static material. Bag it up, seal the bag, and be gentle with it.
Every time you remove a heatsink from an integrated circuit (like a CPU or GPU, for instance), you have to clean off all of the old heatsink compound and apply a new coat. The best way to do this is to get some shop towels (the heavy blue paper towels you find in auto parts stores) and some rubbing alcohol, and use them to get all of the old material off of both the heatsink and the chip. Obviously you would not pour the alcohol onto the chip -- just dab some on a folded paper towel and work with that.
Once it's all gone, apply a thin, even coat of new heatsink compound. Do not use too much. If you find yourself thinking, "Gosh, that might not be enough, I should put more on it," then stop, put the heatsink compound away, and put the heatsink back on. Even if it looks like too little, remember that there is only a tiny little invisible space between the heatsink and the chip; this is what the compound is meant to fill. When you put too much on, it oozes out of the side of the heatsink and can wreck the chip, the board, or the whole computer.
The white compound that comes with most CPU heatsink/fan units (and many other things) will work fine. You don't need the fancy silver stuff, no matter what the reviews say. Yes, it could lower the temperature of the chip by a couple of degrees, but for the silent non-overclocker majority, silver compound is unnecessary. The price difference between the two is not all that great, so you may want to go ahead and get the silver stuff anyway -- it's up to you. Ultimately it doesn't matter which one you use, as long as you use it properly.
No serious technician wears an anti-static strap (unless, of course, company policy demands it). By maintaining a constant awareness of static electricity, you can properly handle any computer component safely. However, it makes life a lot easier when you use anti-static gloves. Not only can you be certain that you will not transfer a deadly static charge to an electronic component, but you'll also save your hands from the cuts and scrapes that you inevitably get from working inside of a computer.
The link above goes to a 12-pair package, which is more or less a lifetime supply. You can find single pairs of Hyflex anti-static gloves if you search Froogle. Remember to get the right size -- I'm a size 9, myself.
Extra cords and cables
I personally have a few Molex-to-SATA power cable converters, a DVI to VGA adapter, a 20-to-24-pin (and vice-versa) ATX power connector adapter, a laptop-to-standard IDE converter (this allows you to connect a laptop hard drive to a desktop computer -- very handy), an extra power cord, two SATA data cables, and two 24" 80-wire IDE data cables in my toolbox. I really ought to have a 36" IDE cable in there in case of emergency.
You never know when you're going to need one of these adapters, cords or cables. When you do need one, though, having an extra is a real life-saver.
The obvious stuff
Then there are the bread-and-butter tools of your technician's toolbox: good Phillips head, flat head, and torx head screwdriver sets in a variety of sizes; needlenose pliers; a flashlight; two-part epoxy; superglue; headphones or earbuds (for testing audio output); a couple of writable CDs; a USB flash drive; and a partitioned screw case.
No matter what you have in your toolbox, the greatest tool that any technician has is his brain. Only experience and a dedication to solving problems will make a great technician. A properly stocked toolbox helps, of course, but if your brain is not properly engaged, all of the tools in the world won't be able to help you fix things.