Civil servants who refuse to return to the office should be paid less than those back at their desks, a minister has suggested.