Am I liable for damages in a work-related auto accident while driving my manager's car?

My manager allowed some of his workers to drive his car on a work related road trip he was not attending. When I drove it, someone backed into me. However, it's a "he said, she said" thing and their insurance is saying that I am liable for damages. Although my manager said he had insurance, the letter I received from the other driver's insurance said that he had none. What are my rights in this case? What I know is that the owner of the vehicle is supposed to be responsible for the damages it causes. Is this correct? I did not file a police report because when I called they said they did not have jurisdiction since it was private property.