I've also gone through and updated the memories. All icon and technique posts have been added.
Dafont is a great resource for downloading fonts. They're all freeware and organized into neat categories. You can find every sort of font you'd want there, from script to pixel to famous fonts from movies and TV shows. Browse it, check it out, download what you want. Eventually you'll get all you want from there and decide you want a specific font or two thata aren't listed there, in which case I reccomend the jack-of-all-trades, the google search.
2. Placement and color
Play with the placement. Just like centered cropping can be dull to the eye, so can centered text. Try it on the side of the image, try tilting it, curving it around part of the image, running it across someone's cheek. Play around and see what works. (though pixel fonts cannot be tilted or warped without looking really bad.) As far as color goes, make sure that you use the eye-dropper tool to pick colors from your image. It makes the text look much more a part of the icon.
Like not using borders and centered cropping, misuse of anti-alias is one of the easiest ways to make your icon look sloppy even if you've put a ton of work into it. TURN IT ON. Photoshop offers four different types of anti-alias: sharp, crisp, smooth and strong. Strong makes the text very clear and easy to read, but sometimes is a bit harsh. Smooth is often very hard to read. I can't tell much difference between sharp and crisp, but those both work well when strong doesn't look right. The only time you don't want to use anti-alias is when you're using pixel fonts, which brings us to...
4. pixel fonts
Pixel fonts, also known as bitmap fonts, are fonts meant to be used at a small size with the anti-alias turned OFF. Each pixel font has a specific size at which it should be used. A fantastic guide of which fonts should be used at which size can be found here. Also, I believe that Dafont will list each pixel font's size with the font. Because they don't have antialias to smooth the edges at a lower opacity, pixel fonts look very opaque. I'll usually counteract that by lowering the opacity of the layer. You should note that I've found that pixel fonts can also be incredibly overused by icon-makers and often aren't the best choice for when you use need a small font.
5. Don't neglect the "boring" fonts
Times New Roman, Arial, Garamond, Minion, etc. All those fonts that you're damn tired of using for papers and reports and stuff (or maybe you use Verdana and Courier to make them seem longer). Well, they're also absolutely fantastic for use on icons. Unlike pixel fonts, they can be used at different sizes. Put different parts of your text in different sizes/widths/heights of the same basic font to add some interest to the text.
6. Mix fonts, but don't go overboard
A mixture of a pixel/plain font with an accent of script can be a great way to add emphasis to one or two words of your text. Don't mix two plain fonts, though, that'll just look strange. Don't mix two fancy fonts, either. It'll look too busy.
7. Script fonts can be gorgeous, but they don't have to be
If your icon has a lot of text, don't put it all in a script font, or it will be illegible and overly busy. Script fonts are best for just a couple of words at a time. Also, be careful with which script fonts you use. Scriptina has been totally overused by novice icon-makers. While the large loops and flourishes can occasionally look nice, they tend to clog the icons and make other people wonder why you're using such an overdramatic and overused font. My favorite script font? Lainie Day, available at dafont. It's got an elegant, but simple handwritten look to it. I also really like Elegant and Claudia for a slightly less fancy look.
8. Make your text legible
Sometimes the text won't be on a solid colored background. To remedy this DO NOT make your text as garish a color as possible so it stands out on both the dark and the light. Instead, use layer effects to create a text layer that can be read on all backgrounds. Using a light font, add a drop shadow in a dark color (100% opacity, 120 degrees, global light, multiple, distance 1, size 0, spread 0). You can also stroke your font with a contrasting color, though you might want to lower the opacity of the stroke to make it more smooth. Outer glow can be nice, especially with script fonts. Sometimes I like to use a dark outer glow (make sure to change the outer glow blend mode from "screen" to something that actually shows up) on a light font and then change the blending mode of the text layer to soft light. I'll usually need to duplicate the text several times (not always with a dark outer glow) to get it to be visible, but it can be nice.
You can also make your text legible by altering the image itself. Using the blur tool, blur the portion below the text a little. This will make the text much easier to read. You can also add a blurry box behind the text. Taking a color from the icon that contrasts with the font color, draw a thick, anti-aliased line at a lowered opacity (anything under 90 will work). Duplicate the layer, move the duplicate down a bit. Merge the layer, and you'll have a line that's opaque in the middle and partially transparent at the edges. Gaussian blur it to a level that looks good then lower the opacity. You probably want to erase the part of the line that's not under the text, but again, play with it.
9. Blending modes
Play with the blending modes on your icon. Hard light can be nice, but sometimes the text turns out too bright a color. Screen is good for light-colored text. Overlay can be a nice effect, as can color or linear dodge. Soft light creates a nice glow with lighter text, but often requires duplicating the layer one or more (or many more) times before the text becomes visible enough to read. I don't use difference much, but I think some people do. Play with the blending modes, and certainly play with the opacity.
10. Rasterize and play
I don't always do this, but sometimes I want my Times New Roman to look a little grungy or my Arial to blur a bit. Rasterize the text layer (once you do this, you can't edit the text, so make sure it's how you want it) and use the filter tools to alter the layer. To make my text somewhat eroded without using a grunge font, I'll go to filter>stylize>diffuse and select "normal." This will make the text absolutely impossible to read, but no worries. Go to edit>fade diffuse (you have to do this immediately- as in not doing anything else in between- after applying the filter or it won't be an option). Fade it down to a very low percentage and the text will be somewhat grungy but legible. Another way to age fonts is by using filter>noise>add noise and adding a bit of noise to the layer. See what you like. Sometimes a simple blur filter (often faded to a low percentage) works nicely. I don't do this with all my icons, but from time to time it can provide a nice change.
These ten steps should cover a lot of common questions and pitfalls about adding text to an icon, as well as providing a few little tips and tricks that I like to use myself. That only leaves one more text related issue: what text to use on your font. Song lyrics can be great, but they can also make your icon look rather silly. I do tend to make icons while listening to music and use a lyric from the current song on my icon. However... I'm often listening to Bob Dylan or other particularly lyrics-heavy music. If you're listening to top 40, try to avoid using those lyrics on your icons. Yes, I'm a music snob and think my music is better than yours. ;-) But, also, lots of other people may well be using the same lyrics on their icons. There's a lot of people out there who will scream if they see another Evanescence lyric on an icon.
You don't have to use lyrics, though the compression of interesting thoughts into a short space does lend itself well. Sometimes I'll stare at an icon and an idea for original text will come to me. It can be a phrase, or just a word. The look on the actor's face, the pose in the picture... they can give me ideas. Oftentimes, nothing comes to me, though. If all else fails, use the character's name or just leave the icon blank.