The problem seems more general than the skinparam backgroundColor.
The following background styling directives (within a note and autonumber) have no effect when generating to SVG
@startuml
autonumber "<back:red><b><size:16> # </size></b></back>"
A->B : Message
note right of A
This is <back:red>RED</back> background
end note
@enduml