What I am looking to find out is the exact date that the penalty no longer applies. Is it the day one reaches 59 1/2, the day after, the first of the month after that date or some other arbitrary date?

I read it as the day you reach 59 1/2.

"Generally, if you are under age 59½, you must pay a 10% additional tax ..."
http://www.irs.gov/publications/p590/ch01.html

I am still in the planning stage, but want to nail down this detail

Well, if I were in the planning stage, I'd probably be looking at how to avoid withdrawing until I had to. Do you really want to withdraw at age 59.5? (There are circumstances where that is necessary - but if it's so you can take the family on a cruise in the Bahamas this winter, maybe you should re-evaluate)