Be prepared to do a lot of walking. It's a great opportunity to discover the place that will be your new home. The first step to finding accommodation is to familiarize yourself with the different neighbourhoods (barrios). Buy yourself a pocket-sized street guide (callejero or guía urbana) and get to know the local bus and metro lines. Explore different areas, talk to people and spend some time walking around. This will help you decide where you want to live.
Once you have decided on some potential areas, the real fun begins. Make sure you know what you are looking for before you start. A list of desired characteristics for your future home is essential, but make sure you split it into "essential" and "nice-to-have" items.
Get your point across in Spanish
If you don't speak Spanish very well or are lacking in confidence on the phone, try to find someone who is willing to help.
Take some time to write down the questions you want to ask in advance. Also, you might find it useful to prepare a few lines that you will use to describe who you are and what you are looking for. Some preparation might save you some trips to see apartments that are very different from what you want.