@Baa video codecs are designed to be decoded forwards. there are 'key frames' which are a full picture and then interframes which are basically a heap of patches to the last frame you rendered. so going forward by one frame is just applying the patches one after another.

going back by one frame requires basically going back until you find the keyframe and then playing it forward again until you get to the frame you want to stop at