What Research Actually Means

Research Is Not Reading and Summarising

Ask most students what research means and the answer comes back the same way: reading up on something and writing down what you found. Read five blog posts on ransomware, pull out the common points, write a summary — that is real work, but it is not research. Not in the sense a university, a journal, or a thesis committee uses the word.

Research is a systematic, disciplined enquiry undertaken to reach an answer that did not already exist in a dependable form. The word doing the real work in that sentence is systematic. Research follows a method that someone else can inspect, and challenge if the method turns out to be weak. Forming an opinion after some casual reading does not clear that bar, no matter how confident the opinion sounds.

It helps to separate research from four things it commonly gets mistaken for. None of these four are bad things to do on their own — they're just not what a supervisor or a reviewer means when they ask "what is your research?"

Often mistaken for research	Why it falls short
Information-gathering	Reading five blog posts on ransomware and summarising them collects information, but nobody's understanding of ransomware has actually moved forward — the summary just rearranges what the original five authors already knew.
Compilation	Copying facts from several sources into one document, even with careful footnotes, is transcription. No new question was asked, no new interpretation offered.
Browsing out of curiosity	Looking things up because you're interested is a genuinely useful habit, but it stops the moment your curiosity is satisfied — not the moment a stated question is actually answered.
A marketing catchword	A product brochure claiming "built on years of security research" is using the word for its persuasive weight. Unless that research is named, dated, and checkable, the phrase is doing advertising's job, not research's.

Six Tests a Piece of Research Should Pass

Controlled. An outcome rarely has one cause acting alone. A study of how a malware family behaves under sandbox analysis has to separate the effect of the malware itself from the effect of the antivirus product, the OS build, and the analyst's own configuration choices.

Rigorous. Every step is chosen because it's appropriate, not because it was convenient. Picking a sample of malware binaries because they happened to be lying around in the lab, rather than because they represent the family under study, fails this test even if every later step is done correctly.

Systematic. The steps follow a traceable order. Collecting network traffic before deciding what question that traffic is meant to answer is not systematic, however clean the data turns out to be.

Valid and verifiable. A result should survive a second look — by the researcher, and by others. A detection rate that cannot be reproduced from the same dataset and the same code fails this test, no matter how good the number looks in a report.

Empirical. Conclusions rest on what was actually observed, not on what seemed likely. "Attackers probably prefer this exploit because it's quieter" is a guess until telemetry from real incidents backs it up.

Critical. The reasoning has to hold up under scrutiny — including the researcher's own. A report that keeps only the test runs that supported the hypothesis, and quietly drops the rest, has failed this test even if every number it does report is accurate.

A study doesn't need laboratory-grade control to count as research — a lot of applied security research can't get anywhere near that. What it needs is honesty about which of the six tests it meets fully, which it meets only partly, and which it can't meet at all. A report that says plainly, "this study could not control for the victim organisation's patch history," is stronger research than one that pretends the problem doesn't exist.

The Four Objectives of Research

Beyond passing those six tests, research generally serves one of four purposes:

• To describe — establishing what exists. A count of how many IoT cameras on a network still use factory-default passwords is descriptive work.
• To explain — establishing why something happens. Why did one phishing template get a much higher click rate than another sent to the same department?
• To predict — building a model that anticipates an outcome before it occurs. Most applied machine learning in security, from malware classification to intrusion detection, sits here.
• To control or improve — using the first three to change an outcome on purpose: a tightened access policy, a patched protocol, a retrained model that raises fewer false alarms.

These four aren't a one-way sequence. A control step usually produces a new phenomenon that needs describing all over again, which is why it makes more sense to think of them as a loop than a straight line. It's worth deciding which of these four boxes a project falls into before writing a single line of code — a project that sets out to explain something but is designed only to describe it will find out the mismatch at the results stage, which is exactly when it's expensive to fix.

Try It Yourself: the WannaCry Timeline

Here's the timeline of one of the most disruptive cybersecurity events of the last decade:

1. The NSA finds the SMB flaw (undisclosed)
2. Microsoft patches it — 14 March 2017
3. Shadow Brokers leak EternalBlue — April 2017
4. WannaCry attacks — 12 May 2017
5. Hutchins finds the kill switch — same day

Notice that the patch existed almost two months before the attack. The research question this raises isn't "what was the vulnerability" — it's "why did patching fail to keep pace."

For each question below, decide whether it's asking you to describe, explain, predict, or control:

1. What share of Windows machines on a given network had applied MS17-010 by 12 May 2017?
2. Why do large organisations take longer to patch a known vulnerability than individual users do?
3. Given an organisation's current patch cadence, how many days would it take to close a newly disclosed critical vulnerability across 90% of its endpoints?
4. What change to patch-management policy would close that gap fastest?

Answers: (1) describe, (2) explain, (3) predict, (4) control.

Check Your Understanding

A simple antivirus vendor's marketing claim — "our product blocks 99% of threats" — is a good test case for the six tests above. Which of the six is it most likely to fail, and why? Think about what dataset the 99% was measured against, and whether anyone outside the vendor could reproduce that number.