Like cold brew coffee, cold brew tea is made by steeping tea leaves in cold milk or water for several hours or overnight.