Assessment refusal to visit

I have a named person at DWP alternative format. I insisted i needed to fill the PIP application online due to my cognitive issues. It wasn't easy but I insisted, after a call from DWP whom them made a "reasonable adjustment" was also given an email address and named person. I could deal with. Reading and being told of the horror stories I think i've been very lucky, it helps knowing the Equality Act also

Was given a date to attend an assessment, called them to say not possible. My Dr. wasn't going to allow that, so appointment was cancelled, however the following week I had a very abusive male call centre say no I'm to aggressive to have a home visit. That made me laugh, WTF, he then said my file is going back "refusal to attend". I made an official complaint to DWP, told them what happened, and they said they would send the file back when they recieved it. All the time I'm emailing my names person who uploaded my letters on my file.

I can honestly say the DWP have been very very good with me, I've been impressed with their attitude, their willingness to help, and they way the explain to me. I have been petrified at the thought of dealing with them

I get a call from the assessor and told no I have to attend, I gave them many alternatives so they could make a reasonable adjustment but they refused but they were nicer this time dealing with me, polite but firm. They said if I do not attend it will be a refusal again.

I wrote to my named DWP person again, saying I'm at my 4 weekly appointment I will speak with the doctor to write a letter. She wasn#t happy with DWP and the assessors and said a letter will be written by the afternoon.

Well next I get a call saying the DWP or assessors have spoken with my doctor and there's no need to collect a letter.

2 days later I get a text from DWP saying they have received my assessment and will let me know of the outcome. WHAT! was this because my file was sent back, but the appointment date was fir another 2 weeks.

I frantically wrote to my named person and call the assessor's (formally ATOS). Apparently I'm not having an assessment they had a long discussion with my doctor instead because they were still refusing to come out and they were told I couldn't attend

has this happened to anyone else? Should I be worried?