The two main beaches on the Gower that are worth visiting are Oxwich Bay, from where you can walk all the way to Three Cliffs Bay (at low tide), and Rhossilli Bay.
There are other good bays, including Pembrey, further along the West Coast.
The tides go out for miles, presentnig walkers with mile after mile of sandy beach. In a different climate you'd have crowds of holiday makers and hotels all along these bays. As it is, they're deserted. Oxwich Bay has one hotel - the 'Oxwich Bay'. Rhossilli Bay has one house - unoccupied because it is so remote. It featured in a Torchwood epsiode.
Delightful wide open spaces, a real gem. It is a good 45 minutes drive to the Gower beaches. Directions are featured on the walks descriptions below.
Swansea itself is not really worth spending too long in. If you do find yourself in Swansea, you can walk along Swansea Bay which IS worth visiting - be sure to avoid the mud flats and don't get caught by incoming tide!
See more details on the Rhossilli Bay walk and on the Oxwich Bay walk.