If you're looking for your father, and you know he's in Japan, your hunt is aided by a few requirements of Japanese residency. For centuries, Japan has kept two types of personal records on all its citizens: one, called a "koseki," logs birth, death, marriage and divorce information; the other, called the "jyuminhyou," registers where each resident--nationals and visa-holders--currently resides. Start with an Internet search that may uncover your father's whereabouts in Japan without having to go through a government source, but if you have to employ the jyuminhyou, it's a fairly straightforward process.
Gather as much personal information about your father as you can. Speak with anyone you know who knows him to glean pertinent data like his full name and date of birth. Both these details are required to begin your hunt, since you'll be searching among millions of names. Try to get a photo if you can, and write down the names of any of your father's known relatives also living in Japan, especially parents and children. Knowing a city or cities where your father's lived in Japan will help immensely as well.
Start on the Internet. It's a valuable tool that may be able to help you quickly narrow your search for your father. Search engines like Google and Yahoo! offer worldwide browsing that allow you to search for your father's name. If it's an unusual name or spelling, or if your father is prominent enough to work for a company with any Internet presence at all, you may be able to confirm his whereabouts with a few simple clicks.
Contact the Japanese Ministry of Foreign Affairs office nearest you by mail or phone, finding the relevant contact information by searching a directory of North American embassies. Explain that you're trying to find your father and ask for a copy of his koseki and/or jyuminhyou. The Ministry of Foreign Affairs will scrutinize and reply to your request. If you receive either document, you should be able to glean (with the help of a translator, if required) your father's current residence. If the only information you're provided is the prefecture (state) or city where your father lives, you'll have a bit more searching to do.
Consult the telephone directory in your father's area (available through an Internet search). If he has a listed number and his name is not too common, he'll be easy to find. You can also contact the government offices for the prefecture or city where your father lives. According to Ancestry.com, this is when working with a Japanese writer or speaker may be handy, depending on whether you choose to write or call. Ask politely for your father's koseki and/or jyuminhyou. Local government offices are where those documents are kept, so you should be able to get information about your father's whereabouts.
Call, write or visit your father after you've located his current contact information. If he has an unlisted phone number, you may have to write a letter or send a local private investigator to his door.