Are you sure you are not in group already or possibly in another instance? Also faster way is simply to find several people to do Skrag Caves with and enter manually as noone uses queue tool.

Yea I'm not in a group just frustrating when it says I am.

thundercooch,
Have someone send you a group invite to be sure you aren't in a group.
I have never heard of anyone getting into Skrag Caves with that queue button. Most go with friends and clan mates. Good Luck!