Not particularly significant challenges though.
The EU is a protectionist bloc with a very high tariff barrier at its perimeter (and a tariff schedule that protected German and French industries but weren't well suited to the UK economy), it's hardly a beacon of free trade.
The UK paid an EU membership fee higher than tariffs for the "luck and privilege" of trading without tariffs.
Furthermore import tariffs are money going in to the UK treasury where they are used to pay for the National Health Service, Education, Social Services, ...