Okay...I'm definitely in.
(just registered)
I really do think it is an excellent idea. You are so right when you say that "we are left in the dark sometimes in this business without feed back". Why do you have a high bounce rate on your homepage? Well, it's hard to corner those people who took one look and left. And if you knew, you would fix it (or try to). So it's a great thing to have others who have no vested interest in your site (friends and family tend to be soooo damn positive and polite...."looks great!" is about all I can ever get from the useless bastards... :
) who will tell it to you gently, but straight. And being developers themselves, can help suggest solutions. And turning a critical eye to your fellow members' sites will help to sharpen your thinking on website design/development on the whole.
It's really win/win, yeah?
I do have one further suggestion...see how we roll already?
I am a member of the alledia.com joomla seo club (love it) and Steve has wisely made the vast majority of the forums on that site private, so that we can all get serious and real about ideas and suggestions without the general public reading our "behind the curtain" conversations.
Imagine someone doing a search on the name of your site, only to end up in a public forum where you are discussing user retention tactics and the problem with security you just pulled an all-nighter to patch. Some things really should not be for public consumption.