I side swiped a car could I have kept going?

I heard a noise then I realize what had happened a few seconds later when I tried to look back in my passenger mirror and it was gone.. I got out of my car and walked back and a guy was standing there. With no hesitation I asked if he was alright and acknowledged it was me who was driving. He thank me for stopping. Because it was a company car we got a police report and exchanged insurance info. half of his driver mirror glass is gone. And all mine broke. Nothing serious. But now I'm wondering what if I'd kept going.