DONT PANIC!! :P
In Douglas Adams’ Hitchhiker’s Guide to the Galaxy, the characters keep traveling back and forth in time, i.e. keep changing their (x,y,z,t). Or, the author keeps juggling them through and in between space-time co-ordinates. Notice that at no point they forget what has happened to them before? Irrespective of who the character is, or when they met each other or even how many times they have gone through the rollercoaster ride!
This would imply that “Recollection” can be considered as another co-ordinate? Say ‘r’.
Then the book is written in (x,y,z,t,r) co-ordinates and moves in the positive direction of ‘r’.