Hi James
The short answer is yes you need to leave it in.
The explanation
If you followed the instructions to the letter, you will have a USB in the rear port which you initialised and created a swap file on. When you restarted the box at that point, the box created 2 folders, one called movies and the other called timeshift.
Everytime you go to a channel, you have the ability to rewind the program back to when you first changed channel. This is called timeshift. Some will say this in not needed and can be turned off, but WooshBuild tires to emulate a sly box and this feature is standard.
The movie folder is where recorded programs are saved
Your swap file helps to keep your box running with enough memory and is stored on the USB.
Picons & EPG data can also be stored on the USB.
Those are the main reasons to have the USB plugged in al the time.