Louise
I'm afraid the simple answer is yes!
Your Fit Note excuses you from seeking work, you still need to participate in what is called Work Related Activity.
You can try contacting your work coach to ask if there is any specific reason that it has to be held at their offices and if not, could it be held over the phone?
Gordon
Nothing on this board constitutes legal advice - always consult a professional about specific problems