Foreigners staying in Japan longer than 90 days need to obtain a Certificate of Alien Registration, also known as an Alien Card. It’s easy to get, but I had to get mine updated several times. If you leave the country without a visa and re-entry permit, they take away your alien card and you have to get another one when you return. Then, when your visa status changes (e.g. you get an extension or finally get your work visa) you go back and they put more stamps on it. I spent a lot of time at the Alien Registration counter in my local ward office in Shibuya.
Around 2007, the government plans to introduce a smart card system. The details of the new system are mostly similar to the current one. Of course, the new cards will be harder to forge. Also, electronic exchange of information will make it easier for the Japanese authorities to keep track of foreigners. Therefore, once these new cards are introduced, you will probably have to be more careful about notifying the authorities about changes to your situation; for example, if you change your job or address.