I keep pushing the idea that participation in convention panels should be open to the public, and the panels pitched SXSW-style: that is, a group of panelists puts together a panel proposal, and then members vote on the panels they want to see. All too often people in the audience (at least some of them) know more about the subject than those on stage.
It's an interesting idea, but for a volunteer organization to implement successfully... pretty challenging. The sausage-making of coordinating the schedule of some several dozen programming participants into a cohesive set of panels, readings and events is an undertaking, especially when done almost entirely out of love. Adding another layer of democracy to that undertaking is not something I would wish on anyone.
That said, the underlying complain has validity. Lord knows that at the conventions I've attended, I've seen some panels where the members really didn't have much business being on it. Hell, I've been on panels where I didn't have much business being on it. At least this time around, my two more problematic panels were subjects I had an interest in, and had done some research on the subject beforehand.
Even if you line up participants-to-panels perfectly, however, any panel that isn't just one person giving a set lecture (and thus... not really a panel) is going to be an organic thing. Unless the moderator rules it with an iron fist, it's going to go where the people on it are going to take it.
Not to mention, any given panel usually has its title and, perhaps, a sentences or two of extra description. The programming coordinator might have had one idea of what a panel is going to be about, while the participants might-- each individually-- have another, and members of the audience something completely different from all of that. Sometimes you just ride the wave where it takes you.
On top of that, when the context is a SF/F/Genre/Lit Convention, I don't think anyone in the audience is expecting or demanding of academic credentials from the panel. They know that the panel consists of spec-fic writers who are mostly doing what they know best: speculating and story-telling. Besides, in my Alcohol In Speculative Fiction panel, I don't think anyone wanted me to drone on about the history of development of beer, wine and distilled spirits.
All that said, there's nothing wrong with just plain bowing out of a panel in which you would truly be out of your depth. I was assigned one originally at ArmadilloCon that I was totally at a loss about, and immediately asked to be taken off it.
Though, to a degree, if you're up on a panel, on some level you are presenting yourself as an "expert", and it's good to do your best to show yourself as one. Or, if nothing else, be up front about your lack of expertise. And be entertaining. That's the thing I most want from a panel, after all.