Changes have been made to 'x' in memory and on disk

Lately after I have made a bunch of changes to files (often via find/replace) and then do something interesting like go to terminal I will get prompted with a ton of 'Changes have been made to 'x' in memory and on disk.' But when I check the changes there aren't any differences. But I can't just blindly click on 'Keep Memory Changes' or 'Load File System Changes' since it's possible that there is a difference and I don't want to lose it. Why does this dialog keep showing up?