Barden Tower dates from the 15th century. It was not built for defence but rather as a lavish residence. The fortified house fell into ruin and was restored in 1658-1659. The remains of Barden Tower are a Grade I listed building. Link
It's interesting to note that the ice cream kiosk is often parked between the double yellow lines and the bridge parapet here. Motorists who do the same are liable to end up with a parking ticket stuck to their windscreen - I know, because I tried it and caught a £30 fine!