If at any time the holder of a blasting permit is, in the opinion of an inspector, guilty of negligence in the execution of his duties, or of any misconduct, or suffers from any infirmity likely to hamper him in the discharge of his duties, such inspector may immediately suspend or cancel such permit.