The Evaluation¶
Please note
This section only provides a rough overview. Conducting a proper evaluation in CS requires extensive and thorough planing and conduct of proper scientific methods.
We only use a rather basic set of methods in this course.
For more in-depth information, consult the relevant literature.
Function¶
Every good and serious research paper needs an evaluation. Neglecting an evaluation in your paper often results in a desk rejection – at least at a serious conference or journal.
-
Links the Approach to the Research Questions
The evaluation links your approach to your research questions, i.e., it analyzes the extent to which your contributions help in answering the research questions. -
Analyzes an Implementation of the Approach
An evaluation requires an implementation of the approach, otherwise it could not be analyzed, since an approach is mostly an idea, concept or logical model that needs some form of manifestation. The implementation is such a manifestation. -
Prerequisite for Discussion
The evaluation usually creates some results (numbers / statistics / texts), the relevance of which wrt. the paper’s problem space and research questions is discussed in the Discussion section.
Types¶
There are two main types of evaluations in CS:
-
Qualitative Evaluation
A qualitative evaluation applies soft measures in order to discuss the extent to which, e.g., a feature is implemented or utilized. -
Quantitative Evaluation
A quantitative evaluation uses statistical tools in order to quantify the “quality” or performance of the implemented approach
Style¶
The same guidelines of the other structural elements also apply to the evaluation.
Structure¶
An evaluation section usually consists of three logical sections or blocks:
-
Explanation of Evaluation Methodology
It starts with an explanation about the evaluation methodology, the data used, sample size, hypotheses, etc. -
Analysis and Results
The second part descibes the execution of the evaluation methods together with the results gathered. This part usually contains statistical figures, data, tables etc. -
Discussion of Results
The last part analyzes (or discusses) the extent of the acquired data and the insights that can be deduced from them. It links the created results with the research questions and problem space and tries to explain the extent to which the gathered results and insights help in answering the research questions. Its intention is to show that the results and deduced insights are valid.Sometimes, the discussion is shifted to a separate section.