I understand that when you are just starting up your own business you have to spend many hours to get it off the ground. However, one of the things you also need to do is replenish yourself and if you have a family as well you need to spend quality time.
You can't take long vacations away from the business but what you can do is make long weekends in the summer longer than 3 days. You can add a day at the beginning and get at least 4 days or you can take a Friday and a Tuesday and make it 5 days of down time.
Now that only means that you utilize 4 weekends of the summer July 1 or 4 (U.S./Canada), August long weekend and Labour Day (September long weekend).
It doesn't seem like a lot but it is a bit of time for you and your family and replenishing both you and your relationship with them.
This also helps you to head into a busy season, the Fall leading up to Christmas when there are built in days off there too.
These are necessary times off and I encourage you to do it.