ESA assessment, can they come to me?

I've been sent an appointment for a face to face assessment for my ESA claim. I was originally put into the support group without an assessment/medical, and my doctors letter stated that there was no prospect of improving and it was likely that I would get worse - so my first question is, should I be worried that I have to have an assessment this time?

Also getting to the appointment is a nightmare, I explained in my form that I get back spasm if I travel for more than 10-20 minutes and its at least a 40 minute drive. I takes me a long time to get up, I have to wait for meds to take effect and getting washed and dressed takes a long time involving rests and I'm not ready to do anything before about midday, but the appointment is for 9am. I rely on my sister to take me to appointments etc and get my powered chair into her car and explained that I cant go anywhere on thursdays or fridays due to her night shifts - they've sent me a thursday morning appointment!
If I try to get to an appointment that far away at that time of day I will be in a right state by the time I get there and it would take days to recover. My brain doesnt function well when my back is spasming, I would really struggle to answer questions and to remember what I'd said.
I've heard that some people have their assessment at home, would I need a doctors note or something? I'm seeing her later today so I want to find out asap - and should I ask her for a letter to support my application?

Thank you, hopefully I should be able to get a letter fairly quickly.

The doctors letter I mentioned in my original post was from my former doctor and supported my original ESA application in 2014, when I was moved over from the previous system - I was unable to get a new letter to support my current application as my new doctor was on holiday, so I didnt have any up to date medical evidence to support the application this time.
Should I also ask my doctor for a letter stating that my health would be adversely affected by any requirement to do work related activities? I saw her shortly after sending the forms and told her that she might be contacted about my ESA application, and she agreed with my previous doctor that I was in no fit state to do anything like that.

I called the assessment centre and was told a number that the doctor can fax to request a home assessment . Should I ask her to include the substantial risk supporting statement in that, so they have the information in advance? Or should it be a separate letter that I give to the assessor when I have the appointment?

Sorry to keep asking things! I am sure I should be covered by some of the descriptors as well as the 'substantial risk' but I want to make sure I've done everything I can and make sure the right people see the doctors information. If I had to do work related activity it would send me into a downward spiral physically and I would be at risk of not getting to appointments, leading to sanctions etc etc ... anxiety isnt even part of my problems but this process is making me very anxious!