If I write a novel that doesn't pertain to my job, can my employer still own the rights to my novel?

Under Virginia law, when analyzing a party's obligations to a contract, you must look to the four corners of the agreement.  In other words, without seeing the entire contract, no one is going to be able to answer the question accurately.  That said, without providing legal advice, I suspect that your contract likely only relates to the subject matter of your employment and that your employer would; 1. not be interested in your fantasy novel, and 2. probably will never know you are writing one.  That said, they could make a claim based on the limited language you have cited.  If, for example, you turn out to be a modern-day JRR Tolkein, they could sue you, claiming you wrote your book on the company dime (and time), and thus the work is owned by the company.  The safest thing to do is get explicit permission in writing from the company in writing, confirming that they relinquish all claims to your particular work of authorship.  Finally, do not forget to file a copyright application for your work.