UML 2.4.1 allows for several "pseudostate" nodes in state machines. The site below has the official UML specifications. The "Superstructure specification" document contains descriptions of all the pseudostates under seciton 15.3.8.
However, this link: http://www.uml-diagrams.org/state-machine-diagrams.html
Is a little less formal than the specification, but also contains sample pictures of what these "pseudostates" look like in diagrams. Look down the page under "Pseudostates." You have already been working on the entry and exit markers as I saw on forum threads.
A "choice" pseudostate looks just like the decision diamond that you support with if/then/else in Activity diagrams. It can have one arrow entering, and multiple arrows leaving it, each labeled with the condition under which they are taken.
A "fork" or "join" pseudostate is just like those in the Synchronization topic on your Activity diagrams page: --> ===B1===
A "junction" pseudostate looks just like the starting point that is currently supported in state diagrams. It can have multiple arrows entering it and leaving it, though. Perhaps using "(*)" might be a good way of specifying it, since [*] is already used for the starting/ending points.
I've not used the history pseudostates before, but they are apparently open circles with "H" or "H*" in the middle.
I found this diagram that shows examples of entry/exit/fork/join/choice/junction