That should be removing itself, just tried it locally with a clean install of BE 2.5 (using default XML storage).
Having said that, clearly it's not for some reason. For XML storage it's the following folder.
It should be referenced there, or there abouts.
I don't have SQL CE set up, but I think it's the same for this.