@Mina
I try to keep all meta information (including tags) in the file itself (caveat: there are at least three different metadata streams supported across image formats: Exif, IPTC, XMP, oh and Photoshop, and even more with raw image formats), and I know how to keep the modified time stamp, if I want.
But yes, it _is_ a hard problem.