Questionnaires, interviews, and sandbox telemetry, and the tradeoff between an instrument's reach and its depth -- a questionnaire scales to 500 employees, an interview does not, but it surfaces what no tick-box question would.
Sign in to mark this article as read and track your progress.
An instrument is whatever the researcher uses to turn an observation into a recorded data point: a questionnaire, an interview schedule, or a sandbox logging script.
Questionnaire. A fixed set of written questions, administered identically to every respondent, usually to produce quantitative data. A well-built item for a security-awareness study reads, for example: "In the past month, how many unexpected emails did you report to IT security? (0 / 1–3 / 4 or more / not sure)." Every option is mutually exclusive, and "not sure" is offered so nobody is forced to guess.
Interview. A structured interview asks the same fixed questions to every participant. A semi-structured interview keeps a core list of questions but allows follow-up on interesting answers. An unstructured interview starts from a broad theme and lets the conversation find its own shape — useful with SOC analysts when the researcher doesn't yet know which questions matter.
The choice between a questionnaire and an interview is really a choice between reach and depth. A questionnaire scales to five hundred employees across three offices; an interview doesn't scale that way, but it can surface a reason an analyst ignores an alert that no tick-box question would have anticipated.
Not every study needs a questionnaire at all. A study comparing how three ransomware families behave once executed uses the sandbox itself as the instrument: a set of scripted API hooks, file-system snapshots taken before and after execution, and a network capture. Its validity depends on whether the sandbox convincingly resembles a real endpoint. Malware that detects a sandbox and refuses to run produces empty data — which is itself a finding worth reporting, not an experiment gone wrong.
A study wants to know why SOC analysts routinely dismiss certain alerts, but the researcher doesn't yet know which specific factors matter. Which interview style suits this stage best, and why would a structured questionnaire be premature here?
