Might as well make this exploit formally public as more and more players are becoming aware of it.
Once you zone into a Dalaran arena match, have at least one player Alt+F4 out of the game. When your teammate calls out that the gate has opened and the match has started, immediately log back in. Since you can still enter and resume your match after a DC, you'll still be in the pipe and won't be forced out. Now just wait it out until the other team gives up. Of course, this is only worth doing at high ratings, where losing takes a bigger hit.
My 3s team managed to pull off 3 wins against Reckful/Venruki/Tenderloinx yesterday with this cheese exploit and Dalaran RNG (didn't bother trying legitimately since they countercomped us).