If you only want to remove one button, the solution is easy:

list = {Button[1], Button[2, list = Delete[list, 2]], Button[3]};
 
Dynamic@list

If you want to remove multiple unique buttons, you have to use some kind of identification for each button other than the actual position in the list, as that is changed when one of them is removed. Here I use a $position \rightarrow button$ identifier:

list2 = list2 = (# -> Button[#, list2 = DeleteCases[list2, _[#, _], 1]]) & /@ Range@4;
 
Dynamic@(Last /@ list2)

If you only want to remove one button, the solution is easy:

list = {Button[1], Button[2, list = Delete[list, 2]], Button[3]};
Dynamic@list

If you want to remove multiple unique buttons, you have to use some kind of identification for each button other than the actual position in the list, as that is changed when one of them is removed. Here I use a $position \rightarrow button$ identifier:

list2 = (# -> Button[#, list2 = DeleteCases[list2, _[#, _], 1]]) & /@ Range@4;
Dynamic@(Last /@ list2)

After v10, it is even simpler with associations:

a = AssociationMap[Button[#, a = KeyDrop[a, #]] &, Range@4];
Dynamic@Values@a