This artificial beach created in 2000 and has been popular with the locals on sunny day ever since. The lagoon and the hot tubs are heated up with the hot water from the reservoir tanks situated close by.
I'm not so sure that your initial idea for the travel to Iceland included hanging out at the beach and swimming in the waters. It does sound crazy, right? In the most preferable circumstances the seawater in the lagoon is around 15°-19°C and the hot tubs are around 30°-39°C hot.
Accept the challenge and go with some crazy locals to swim in the non-heated area.