A stipulation or an agreement for an order appointing a referee does not constitute a waiver of grounds for objection to the appointment of a particular person as referee under Code of Civil Procedure section 641. Any objection to the appointment of a person as a referee must be made with reasonable diligence and in writing. The objection must be served on all parties and the referee and filed with the court. The objection must be heard by the judge to whom the case is assigned or by the presiding judge or law and motion judge if the case has not been assigned.