The science of dungeons

I am currently filling out a survey about a game I play - Dungeons and Dragons.  Many of the questions on the survey were reasonable to my mind, as they asked about things like how well I fit with the group, what sort of player I am, etc.  I think the creator of the survey will be able to get interesting data out of it.

But some of the questions are badly written at best.  For example, there is a question asking about how often I get certain feelings.  Do I feel like 'humans are basically good' once / week, 2-3 times per week, or every day?  What?  I don't suddenly get struck with a 'humans are good!' feeling and note that down.  I certainly can't give you a rate per week of such things.  But the survey doesn't have a 'this doesn't apply to me' answer, so I have to click a rate in order to move on.

It also had a really silly question about how long it takes me to feel like I belong in a gaming group.  Is it 1-2 sessions?  3-4?  5-6?

How in the world are you getting any kind of real data out of this?  I don't join a new group every month and then count how many sessions is it until I get a 'belonging' feeling.  When I joined my last group I got along with the people fine.  We have had lots of sessions, but I certainly never got a 'I belong' feeling.  When joining my previous group I knew most of the players for a decade or more so clearly I wasn't going to suddenly feel like I belong after 4 sessions - that sense of being comfortable with those people was already there.  Really, do we actually expect that people who click 7-8 sessions are in any way different from those who click 5-6 sessions?  This is a complete guess, with people clicking randomly.  The players and the game style matter in this way, the number of sessions does not.

What I actually want is an answer that says '1 or infinity, depending on if the group is people I like or not'. 

It feels like answering a question 'Do you like jobs?' without any context.  Sure, you will get answers, but the information given is so vague that the answers will be totally random and worthless for digging out useful data.  I like some jobs, but I hate others, just like nearly everybody.

The design of scientific experiments and surveys is critical.  When you build surveys foolishly, or design experiments badly, you end up falling prey to Garbage In, Garbage Out.  All the fancy statistics in the world won't fix your conclusions when your data is rubbish at the outset.

I think this is something we need to train people on more effectively.  It isn't easy though, because you often have to look at things from somebody else's point of view, and you don't even know what somebody that might be.  There might well be people who feel 'humans are good' intensely 3 times per week, and that question works perfectly for them.  However, there are going to be a lot of people who just click randomly because the question is meaningless to them, and if you don't have a 'this doesn't apply' answer your data will be corrupted.  You also need to account for cultural differences and language proficiency when you write your questions.

Writing good questions and setting proper controls to make your experiment give you the information you actually want is hard, but it is critical if we don't want our knowledge corrupted and diluted by all kinds of studies that don't actually show what they claim to show.  We already have enough of that, thanks.