Acceptance Criteria

Acceptance of a submission to TMLR should be based on positive answers to the following two questions.

Are the claims made in the submission supported by accurate and convincing evidence?

This is the most important criterion. This implies assessing the technical soundness of the claims made Any gap between claims and evidence should be addressed by the authors. Often, this will lead reviewers to ask the authors to provide more evidence by running more experiments. However, this is not the only way to address such concerns. Another is for the authors to adjust (reduce) their claims.

Would some individuals in TMLR’s audience be interested in the findings of this paper, and does the paper communicate those findings clearly to its intended audience?

This is arguably a more subjective criterion, and therefore needs to be treated carefully. There are two parts to this criterion, which we discuss in turn.

The findings in a submission should be of interest to some individuals in TMLR’s audience. Generally, a reviewer who is unsure as to whether a submission satisfies this criterion should assume that it does. Crucially, it should not be used as a reason to reject work that isn't considered “significant” or “impactful” because it isn't achieving a new state-of-the-art on some benchmark. Nor should it form the basis for rejecting work on a method considered not “novel enough”, as novelty of the studied method is not a necessary criterion for acceptance in TMLR. We explicitly avoid these terms (“significant”, “impactful”, “novel”), and focus instead on the notion of “interest”. If the authors make it clear that there is something to be learned by some researchers in their area from their work, then the criterion of interest is considered satisfied. TMLR instead relies on certifications (such as “Featured” and “Outstanding”) to provide annotations on submissions that pertain to (more speculative) assertions on significance or potential for impact.

Additionally, a submission should clearly communicate what it has found, why those findings may be of interest to some researchers in the area, and what readers can learn from the work. The writing and organization should make the paper's main contributions, findings, and takeaways understandable to TMLR's (human) audience. Problems of presentation should be evaluated in terms of whether they materially impede a reader's ability to understand the work and its relevance.

Here is an example of how to use this criterion. A machine learning class report that re-runs the experiments of a published paper has educational value to the students involved. But if it doesn't surface generalizable insights, it is unlikely to be of interest to (even a subset of) the TMLR audience, and so could be rejected based on this criterion. On the other hand, a proper reproducibility report that systematically studies the robustness or generalizability of a published method and lays out actionable lessons for its audience could satisfy this criterion. Similarly, a submission may contain potentially useful findings but present them so unclearly that (human) readers cannot determine the problem setting or experimental setup, in which case it will not meet this criterion. If a paper fully is AI generated with little or no human involvement, then at least with current AI systems, it is unlikely to meet this criterion.