As a general rule, you must have lived in Japan for a period of ten years before you are eligible to apply for permanent residency. If your spouse is Japanese, however, this period can be as little as three years.
You also must have a visa valid for the entire length of the application process (usually several months).
In order to have any chance of obtaining permanent residency you cannot have a criminal record (some officials extend this to traffic violations). In addition, you should be able to offer some contribution to Japan: working as a diplomat or for a listed Japanese company qualifies, as does teaching at a university. It helps if you have received professional awards or honours.
Be prepared to provide a convincing written explanation listing the reasons for your application, along with detailed financial records. Your employer will also have to submit financial information as part of the process.
Permanent residents no longer have to renew their visas to remain in Japan, though they must still apply for re-entry permits before travelling outside the country. They have easier time dealing with Japanese banks when it comes to credit and loans, and in some parts of Japan they are even qualified to vote in local elections.
A note on marriage, citizenship and residency
If you marry in Japan and obtain citizenship or residency while married you will keep that status even if you divorce (if you have a spouse visa as a result of marriage to a Japanese national your visa will remain valid until its listed expiration date). Be forewarned, however, that foreigners are not well-served by Japanese courts during divorce proceedings. You should therefore avoid using marriage solely as a means of obtaining permanent residency or citizenship, even though divorce will not affect your status of residence.