On several occasions while camping for the night the party has seen unexplainable red blurs. The blurs have been associated with party member disappearing for a short time.
Update: Recently Bigby noticed a red blur that seemed to "deposit" Ostler as it streaked by. This is the first conclusive evidence the party has that the red blurs are responsible for moving characters.