Our solution to the mobile data usage in Japan dilemma has been SoftBank's Global Rental unlimited plan, which costs 1550 JPY per day ($15). It is expensive, but that's par for Japan's mobile access for temporary use. The ease of being able to use our own phone without worrying about data cost is also a convenience we deem worth the premium rate.
SoftBank's stores are ubiquitous in Japanese cities, but you'll have the most interaction with their airport locations, where the staff is knowledge, speaks (good enough) English, is used to answering questions regarding foreign phones, and has clear signs explaining each available plan. All of their locations for Global Rental are on their website, and their Tokyo-Haneda International Airport location is pictured above.
Assuming your phone is newish and 3G/4G/LTE-compatible, you'll most likely be able to use it on a Japanese network, like SoftBank. It must be unlocked! If your phone is not unlocked, the staff will point you to their MiFi rental, which is slightly cheaper than the SIM card rental and allows you to carry a personal WiFi hotspot device.
All told, our 4-day SIM card rental on the unlimited data plane totaled 6440 JPY + 515 tax ($62 + $5).
Word from our friend @thatjohn is, there's another option out there which rings in cheaper than SoftBank's plan.
We haven't tried it ourselves yet, but it's definitely worth looking into, since 30 days equals only 4100 JPY ($40). The deal comes via econnectjapan.com, who also offer a SIM rental of 1GB for the same price, for use up to 30 days, in case your trip is far shorter.