Lol I know this is like almost two years old but you can leave the rat cave. Just interact with the boss, run away, then go back to the entrance and someone will let you out. There's also an NPC that will give you an item that makes the fight really easy.