Solution is to make a fantasy-version of a torpedo. Create a large anti-ship battering ram from suitable bone mass or rock, attach handles to the ram and then have the ram driven directly into the underside of the boat by a host of merfolk.

Ship hull is optimized to deal with water, not massive merfolk-powered rams.