Actually I don't think he ever got around to actually returning the sword to his back, no... he brandished it when he went outside during the attack of the crows and has been in danger constantly since then, so I think he's still got it in hand. Oh, but the standard cleaver-type runeswords are indeed generally kept on the back (and would have been by this rune mage as well) due to them being simply too large to keep anywhere else.
Momentum is indeed still an issue when using the massive runed cleavers, and it [I]does[/I] tire a rune mage to carry around their sword; especially trying to hold it like Thomas did when he was threatening Ixion, or alternatively how he held it in the novel when he released the lightning-rune (which is why they usually support their wielding arm with the other one in these cases), and generally when being held in a somewhat horizontal manner without resting on something else. Holding a runesword passively one would usually either rest the blade against the ground or one's shoulder, since keeping it raised for any extended duration is quite strenuous. Additionally I should point out (although you probably already know this) that rune mages do feel the weight of their runeswords as long as they keep them on their back due to the [I]Unity[/I]-rune not being activated during this time, since they aren't touching the energy-conveying crystal handle.
As to the exact device serving to hold the swords in place on the rune mages' backs, it is a contraption of several reinforced leather straps that ties around the mage's shoulders and chest on one side and locks with the spike-like crystal extremities of the runesword hilt on the other; normally one would have to undo the straps to actually draw the runesword (which can be done by releasing a clasp, which can be reached while grasping the actual handle), though in urgent cases it is possible (but not guaranteed to succeed) to get the sword loose with some violent pulling and wriggling due to the shape of the sword-hilt... though this method does run a risk of damaging the harness [I]and[/I] the sword-hilt. Sitting with a runesword is uncomfortable but possible, since it is only anchored to one point between the rune mage's shoulderblades and can be shifted around to various angles as this necessitates, whereas running with it can be difficult and even dangerous, so one usually doesn't do that without at the very least reaching up and grabbing the hilt before running, to have some control over where the blade is actually pointing.