He is as daft as a brush, “New brooms sweep clean“ or “I got the brush-off”. The cleaning instrument brings order (to the house, to your life) and it sweeps away rubbish and dirt (or worries). Symbol of superstition and magic, e.g. the witches' broom.