It shall be lawful for a pastor of any religious society in this state to join together in marriage such persons of the society to whom a marriage license has been issued, according to the rules and customs established by the society. The clerk or keeper of the minutes, proceedings, or other books of the religious society wherein such marriage shall be had and solemnized, shall make a true and faithful register of all marriages solemnized in the society, in a book kept by him for that purpose, and return a certificate of the same to the clerk of the circuit court of the county, to be by him recorded, under the penalty prescribed in Section 93-1-21.