When I was developing my hill climbing bike, I used to spend hours working out the best way to save weight on a bike. I developed a system for calculating the cost per gram saved. Among the best things to change was buying a lightweight saddle and seat post. This offered the best way to save weight for the least money. The most expensive way to save weight would be to do something like upgrade from an Ultegra groupset to a Dura Ace groupset. Here the cost is very high and the weight saving low. Make sure you prioritise and remember to do the most efficient changes first.
Seat Post:
Deda Elementi Ultra Seatpillar weight 260g only £23.99
Saddle:
Selle Italia SLR XP £53.99 weight 190 grams link
Wheelset
Mavic Ksyrium SL Wheel Set £527.40 weight 1485g link
Groupset
Shimano Ultegra offer best value. Shimano DuraAce will save a few more grams but is comparatively more expensive
Pedals
The lightest are Speedplay pedals. Speedplay Titanium come in at 205 grams cost about £125 link
Frame
Carbon Fibre offer a lightweight range of frames, but, some Aluminium and Scandium frames are not much heavier and are generally cheaper. Having said that carbon fibre frames seem to be coming down in price.

First place to look at saving weight is to the rider, most could lose a stone and not notice it except when climbing when it becomes very apparent. I’ve known riders fitting titanium bolts, carbon bottle cages when they are blatantly overweight. Do you need two 750 size bottles? Recently passed riders in the Iron mountain sportif climbing the ‘tumble’ on crbon bikes with Camelback type water carriers humped on their backs.