Is the battery/tray earthed to the car correctly?

Had a similar issue where this was the problem, car would drive fine and start after a drive but was lazy to start after a period of time of non use had passed. The alternator was not charging the battery correctly while driving.

Hope you get sorted.