Yes, Seljavallalaug is worth the detour if you want a quiet valley, a historic mountain pool, and a stop that feels less polished than the major South Coast sights.
The important caveat is that Seljavallalaug is not a spa substitute. It is a simple concrete pool in a steep green valley, reached on foot, with conditions that can feel rough, slippery, cold, or peaceful depending on the day.
Add it when your South Coast day already connects Seljalandsfoss and Skógafoss and you want one quieter, stranger, more local-feeling stop between the big names. Skip it if your schedule is tight, the weather is messy, or everyone in the group expects managed bathing comfort.
Add Seljavallalaug for travelers who enjoy imperfect places and have enough slack to walk in, look around, and decide on site. They would leave it out of a rushed first South Coast day where Kvernufoss, Dyrhólaey, and Reynisfjara are already competing for time.