For me, since this application is portable (in opposition to installable applications), I prefer that all files would be in app's directory.
Pros: application is truly portable, so it can be on USB flash drive, in cloud etc. Version controls is also good thing: all application's versions are completely independent.
Cons: every time after new version is launched, I must configure it anew.