English in Korea my original goal was to teach in Japan. After all, Japanese was my major in college and I've spent quite a bit of time there. However, like the Rolling Stones said, "You can't always get what you want." Although I actually had a job offer on the table at one point (from G.education Nova), I wasn't able to find a job that fulfilled all of my requirements (namely, a decent salary) and so I set my sites elsewhere.
It's pretty hard to get a good job in Japan right now. From what I've seen online, the market there is rough for prospective English teachers with no previous experience. More and more people are falling in love with Japan and going there hoping to teach, while fewer and fewer positions are being made available as schools scale back or even close. Schools that are hiring want people who are already in Japan and don't require visa support. Those that are willing to sponsor you for a visa want teaching experience. It's an employer's market so schools can afford to be picky. That being said, if you're willing to work for low pay for a year or so to gain experience there are still jobs out there. (This option wasn't available to me as I have student loans to pay off.)
As with Korea, there are two main types of English teaching jobs in Japan: public and private schools. At a public school, you're what is called an ALT, an Assistant Language Teacher. You co-teach English with a Japanese teacher at a public school. ALTs are provided by private companies, like Interac and Altia Central, or the government, which is what JET is. There are also private schools, known as eikaiwa, conversation schools. Unlike Korean private schools, which largely use recruiters to find employees, private Japanese schools hire directly.
So what do you need to be able to teach in Japan? A college degree is absolutely necessary. You don't have to speak Japanese, although it doesn't hurt. A TEFL certificate is not necessary but could give you an edge over non-certified applicants. And, as I said before, if you already have a visa eg you're married to a Japanese person then you're good to go.
Get your resume together and rewrite it with an ESL focus. Bagged groceries at Safeway? Great, but even better if you tried to teach your Spanish-speaking co-workers some English on your break. Prepare a nice cover letter too. Have a passport photo taken (and wear a collared shirt and tie). You'll need that for your application. Get two former bosses to write generic letters of recommendation for you. Then scan everything, including your diploma and any TEFL certificates. It also wouldn't hurt to write a short essay on why you want to teach in Japan. You can always add in specifics like location when applying.
Should I Stay Or Should I Go?
You don't have to be in Japan to get hired. There are lots of job sites (below) to help you do that. However, being in the country could give you an advantage. That being said, Japan is a very expensive place to be and it could take 3-4 months to land a job so you'll need to being a lot of money with you, something like $5000.
Here's a list of some of the larger eikaiwa chains, some of which have offices outside of Japan where you can interview in person. Others will want to interview via Skype or phone. Eikaiwa often pay better than public schools but your hours will vary and be when people are not working or in school, so evenings and weekends.
G.education Nova (Geos)
James English School
If you'd rather work in a public school with normal 9-5 hours, try one of these.
There are many other, smaller schools also hiring. Look on the following job sites for job postings. Gaijinpot is the biggest and best known, but not every school will post there. Also, everyone looking for a job goes there so there are literally hundreds of applicants for any one job. Check the sites everyday and if you can, check at the end of the day, as that's when new ads tend to go up (it's morning in Japan then).
TEFL Jobs Now
Japan English Teacher
You Can Teach English
ESL Job Feed
Dave's ESL Cafe International Job List
All About Teaching English In Japan
Jobs In Japan
Kansai Scene Classifieds
Kansai Flea Market Classifieds
Kansai Free Ads
Check these boards for additional information. As with any expat boards take what people say with a grain of salt.
Dave's ESL Cafe International Forums
I Think I'm Lost