Sorry, I was not clear. I am indeed storing file offset (std::streamoff) in a vector in memory, and not writing a files containing offset. And I definitively agree that this should be faster than writing an offset file =).
But I still need to read through the file (but not parse everything) once to locate these offset when loading the file.