Don't worry. You will remember them if you practice on a regular basis on the Pīnyīn chart :)
Furthermore, we should be practicing tones always along with the other topics. It is an omnipresent element of the Chinese language.