History of JMLR

JMLR was founded (incorporated as a non-profit) in 2000 by Leslie Kaelbling, Computer Science Professor at MIT, with the goal of starting a new journal in the area of machine learning whose contents are freely available to the research community at large via the web. Inspired by Steve Minton's founding of JAIR, Leslie assembled a founding team, and with the operational help of David Cohn, version 0 of the jmlr web site was up and running on www.jmlr.org on June 23, 2000. At ICML shortly thereafter, MIT Press, who had agreed to do the printed copies of JMLR, were handing out flyers announcing the journal (it turned out later that MIT Press were losing too much money on JMLR, leading to termination of the publishing agreement as of end of 2004. JMLR then went with Microtome, a small press that published JMLR annually, at a rather low subscription price.)

On 8/8/2000, JMLR was announced on a number of mailing lists, with the following message:

Announcing the JOURNAL of MACHINE LEARNING RESEARCH

The Journal of Machine Learning Research (JMLR) provides an international forum for the electronic and paper publication of high-quality scholarly articles in all areas of machine learning. JMLR seeks previously unpublished papers that contain:

new algorithms with empirical, theoretical, psychological, or biological justification;

experimental and/or theoretical studies yielding new insight into the design and behavior of learning in intelligent systems;

accounts of applications of existing techniques that shed light on the strengths and weaknesses of the methods;

formalization of new learning tasks (e.g., in the context of new applications) and of methods for assessing performance on those tasks;

development of new analytical frameworks that advance theoretical studies of practical learning methods;

computational models of data from natural learning systems at the behavioral or neural level; or

extremely well-written surveys of existing work.

JMLR has a commitment to rigorous yet rapid reviewing; reviews are returned within six weeks of paper submission. Final versions are published electronically immediately upon receipt, and an annual paper volume is published by MIT Press and sold to libraries and individuals. JMLR is accepting new submissions. Please see http://www.jmlr.org for submission information.

Accepted papers, when published on the web, will be announced via an email list. To subscribe, send an email message to majordomo@ai.mit.edu with body text "subscribe jmlr-announce" (without quotation marks).

Editor: Leslie Pack Kaelbling

Managing Editor: David Cohn

Action Editors:

Peter Bartlett, Australian National University, Australia

Craig Boutilier, University of Toronto, Canada

Claire Cardie, Cornell University, US

Peter Dayan, University College, London, UK

Thomas Dietterich, Oregon State University, US

Donald Geman, University of Massachusetts at Amherst, US

Michael Jordan, University of California at Berkeley, US

Michael Kearns, AT&T Research, US

John Lafferty, Carnegie Mellon University, US

Heikki Mannila, Helsinki University of Technology, Finland

Fernando Pereira, Whizbang! Laboratories, US

Pietro Perona, California Institute of Technology, US

Stuart Russell, University of California at Berkeley, US

Claude Sammut, University of New South Wales, Australia

Bernhard Schoelkopf, Microsoft Research, Cambridge, UK

Larry Wasserman, Carnegie Mellon University, US

Stefan Wrobel, Otto-von-Guericke-Universitat Magdeburg, Germany

Editorial Board:

Naoki Abe, NEC Corporation, Japan

Christopher M. Bishop, Microsoft Research, UK

Andrew G. Barto, University of Massachusetts, Amherst, USA

Henrik Bostrom, Stockholm University/KTH, Sweden

Carla Brodley, Purdue University, USA

Nello Cristianini, Royal Holloway, University of London, UK

William W. Cohen, Whizbang! Laboratories, USA

David Cohn, Burning Glass Technologies, USA

Luc De Raedt, University of Freiburg, Germany

Saso Dzeroski, Jozef Stefan Institute, Slovenia

Nir Friedman, Hebrew University, Israel

Dan Geiger, The Technion, Israel

Zoubin Ghahramani, University College London, UK

Sally Goldman, Washington University, St. Louis, USA

Russ Greiner, University of Alberta, Canada

David Heckerman, Microsoft Research, USA

Thomas Hofmann, Brown University, USA

Tommi Jaakkola, Massachusetts Institute of Technology, USA

Daphne Koller, Stanford University, USA

Michael Littman, AT&T Research, USA

Sridhar Mahadevan, Michigan State University, USA

Yishay Mansour, Tel-Aviv University, Israel

Andrew McCallum, Whizbang! Laboratories, USA

Raymond J. Mooney, University of Texas at Austin, USA

Stephen Muggleton, York University, UK

Foster Provost, New York University, USA

Dana Ron, Tel-Aviv University, Israel

Lawrence Saul, AT&T Labs, USA

John Shawe-Taylor, Royal Holloway, University of London, UK

Dale Schuurmans, University of Waterloo, Canada

Yoram Singer, The Hebrew University, Israel

Alex Smola, Australian National University, Australia

Padhraic Smyth, University of California at Irvine, USA

Moshe Tennenholtz, The Technion, Israel

Sebastian Thrun, Carnegie Mellon University, USA

Naftali Tishby, Hebrew University, Israel

David Touretzky, Carnegie Mellon University, USA

Chris Watkins, Royal Holloway, University of London, UK

Robert C. Williamson, Australian National University, Australia

Advisory Board:

Shun-Ichi Amari, RIKEN Brain Science Institute, Japan

Andrew Barto, University of Massachusetts at Amherst, USA

Thomas Dietterich, Oregon State University, USA

Jerome Friedman, Stanford University, USA

Stuart Geman, Brown University, USA

Geoffrey Hinton, University College London, UK

Michael Jordan, University of California at Berkeley, USA

Michael Kearns, AT&T Research, USA

Steven Minton, University of Southern California, USA

Thomas Mitchell, Carnegie Mellon University, USA

Stephen Muggleton, University of York, UK

Nils Nilsson, Stanford University, USA

Tomaso Poggio, Massachusetts Institute of Technology

Ross Quinlan, University of New South Wales, Australia

Stuart Russell, University of California at Berkeley, USA

Terrence Sejnowski, Salk Institute for Biological Studies, USA

Richard Sutton, AT&T Research, USA

Leslie Valiant, Harvard University, USA

Stefan Wrobel, Otto-von-Guericke-Universitaet, Germany"

In addition to finding the volunteers to run JMLR, work in 2000 focused on getting a first slate of strong papers lined up. This included both papers submitted to JMLR as well as some that were transferred from the Machine Learning Journal (upon the request of the respective authors). The first issue is available at https://www.jmlr.org/papers/v1/ and includes several papers co-hosted with www.kernel-machines.org, a website that partnered with JMLR to solicit submissions from the growing field of kernel methods.

On March 27, 2001, a partnership between SPARC (an alliance of universities and research libraries supporting increased competition in scientific journal publishing) and JMLR was announced (https://www.jmlr.org/sparc-pr.html).

On March 30, 2001, the first paper copies of Vol. 1, No. 1 arrived.

At the time, the flagship journal of the field was the "Machine Learning Journal (MLJ)" (at the time, published by Kluwer). There had been discussions with Kluwer about pricey subscription fees and closed access. This culminated in a resignation of most of the editorial board of the journal in 2001 to join and endorse JMLR (http://sigir.org/files/forum/F2001/sigirFall01Letters.html, https://www.jmlr.org/statement.html, see also Donald Knuth's letter at https://www-cs-faculty.stanford.edu/~knuth/joalet.pdf). Resignations started in July 2001, leading to discussions at a Dagstuhl workshop where several MLJ editorial board members were present (Bob Williamson, Manfred Warmuth, Chris Williams, Zoubin Ghahramani, Peter Bartlett, and Bernhard Schölkopf). As a result, JMLR invited all editorial board members of MLJ to also join JMLR. In August, Michael Jordan coordinated a letter announcing a joint resignation

Following this, JMLR quickly established itself as the new flagship journal of the field, a position that it has held until now, although entirely run by volunteers.

In September 2003, Foster Provost reported on the ISI Journal Citation Reports:

The category Computer Science, Artificial Intelligence has 74 journals.

In the 2002 listing for that category, JMLR is ranked #1[...]

In 2004, a new review management software was put into production, written by Christian Shelton (and still in use today!), who had taken over as managing editor from Pablo Cohn in 2003.

In February 2006, Neil Lawrence proposed for JMLR to host online proceedings from workshops that were related to machine learning. Initially, there was skepticism, but when Geoff Hinton proposed a related idea (Curiously, suggesting the name 'Transactions in Machine Learning Research' which is now going to be used for something else), it took off, launching in early 2007 as JMLR: Workshop and Conference Proceedings .

In 2007, there was a proposal by Cheng Soon Ong, Mikio Braun and Soeren Sonnenburg to start an open source software section of JMLR, accepting rolling submissions, much like the kernel machines section. The section started with a position paper written by Soeren et al.: https://www.jmlr.org/papers/volume8/sonnenburg07a/sonnenburg07a.pdf

In late 2007, a committee was put together to find a successor for Leslie, identifying Lawrence Saul as the top candidate. He started to work alongside Leslie in mid 2008, and took over completely in 2009.

In late 2012, Leslie contacted Bernhard Schölkopf to take over as EiC. He argued that with the growth of the field, the job should be shared among two co-EiCs, and in March 2013, Kevin Murphy and him were announced as taking over from Lawrence Saul. They took turns month by month in assigning papers, and introduced a mechanism to make sure the queue gets cleared on time: each EiC needs to completely empty the queue at the beginning of the month. If one does not manage to do it on the 1st, then the queue continues to grow, so there is a strong incentive to do it on time..

In May 2013, JMLR's web space reached its quota (20GB), meaning no one could submit new papers until the quota was increased. ;)

In June 2013, JMLR agreed to publish a special issue with papers from the new ICLR conference, with action editors Aaron Courville, Rob Fergus, and Christopher Manning. Later that year, a model was initiated where top tier conferences could have a 'JMLR track' directly accepting submissions into JMLR. This was subject to some conditions, e.g., that there is enough time for one round of revisions, and that the area chair handling the paper is also a JMLR AE.

In 2015, JMLR renamed the Workshop & Conference Proceedings into PMLR, and announced the plan to turn MLR into an umbrella for several publication ou tlets. Ideas included a PLOS-ONE style journal connected to arxiv.org, a subsidiary that specializes in special issues, and a monthly issue which is a little less formal and more topical.

In early 2017, JMLR changed its publication agreement to CC-By (https://jmlr.org/forms/jmlr-license-agreement-2017.pdf).

In spring 2017, Kevin Murphy stepped down. A new search ensued, and Dave Blei replaced him in late 2017, with Francis Bach joining in early 2018.

In December 2017, many of the JMLR editors contributed to and signed a statement (drafted by Neil Lawrence and Tom Dietterich) about the planned 'Nature Machine Intelligence' journal: https://openaccess.engineering.oregonstate.edu/

In 2020, the full jmlr.org website (minus submission system) was migrated to github. This provides both a backup and a history of the journal website.

In December 2021, Bernhard Schölkopf retired from his position as JMLR Co-editor-in-chief.

Appendix: The following article appeared in the Boston Globe:

_________

SPREADING THE GOSPEL OF FREE SCIENCE

Author: By Nicholas Thompson, GLOBE CORRESPONDENT Date: 11/20/2001

Page: C1 Section: Health Science

Leslie Kaelbling hardly looks like a threat to the scientific establishment.

The perky MIT professor rides to work on a bicycle that neatly folds into a

briefcase-sized rectangle and spends her days trying to make machines that

can learn. As associate director of MIT's Artificial Intelligence

Laboratory, she is surrounded by quirky creations, from tiny fish-like

robots made partly from frog tissue to robots that look like humans and, one

day, may think like them, too.

But Kaelbling leads another revolution in her spare time. As editor of the

upstart Journal of Machine Learning Research, Kaelbling offers some of the

latest debates and developments in artificial intelligence to anyone with

access to the Internet - free. By contrast, the scholarly journals that

largely set the world's science agenda sometimes charge more than $1,000 a

year for a subscription.

Kaelbling would be just another quixotic idealist with her unpaid staff -

the journal "doesn't even have a bank account right now," she acknowledges -

except for one thing. Thirty-nine board members of the leading artificial

intelligence journal in her field, Machine Learning, announced their

resignations last month to join her crusade.

Kaelbling and her supporters argue that putting all top current research

online could well inspire crucial insights from people who wouldn't

otherwise have access; a statistician in Mongolia may spur the next

breakthrough, after all.

"The only thing we care about in the world is that people read our work,"

Kaelbling said.

The rivalry between Kaelbling's journal and Machine Learning - which costs

$1,050 a year for institutions and $120 for individuals - is part of a much

broader debate about how much scientific information should be free. For

instance, the federal government's Human Genome Project and private Celera

Genomics have locked horns repeatedly over Celera's plan to withhold some

key information from the human genetic blueprint that could be sold to

pharmaceutical companies looking for potential new drugs.

More closely parallel to Kaelbling's work, the National Institutes of Health

have created a database called PubMed Central intended to allow anyone in

the world to freely search and retrieve the full text of any published

scientific article, with archives extending back for decades. Several months

ago, a coalition of about 30,000 scientists, led partly by former NIH

director and Nobel laureate Harold Varmus, pledged to boycott any journals

that didn't meet PubMed Central's standards for freely distributing data.

The coalition backed off a little in early September, but the conflict still

looms.

The free, Internet-based Journal of Machine Learning Research challenges an

elaborate system of disseminating scientific information that touches

everything from what appears on the nightly news to which researchers become

stars. A few well-known journals such as Science, Nature, and the New

England Journal of Medicine considerably influence scientific discussion and

have a near hammerlock on determining what science appears in the mainstream

press. A slew of others - including Machine Learning - shape their

respective subject areas, helping determine who gets tenure, where grants

go, and how their fields move forward.

Kaelbling and others note that the scientific community differs in several

ways from journalism, where support for free online access to magazines and

newspapers has withered. For one, most scientific authors don't get paid for

publishing, even in the journals with costly subscription prices. Instead,

they receive their funding from universities, corporations, or government

grants, and publish mainly for prestige and to advance their fields.

Kaelbling earns her salary from the Massachusetts Institute of Technology

and part of her job description requires her to offer public service to the

community, such as editing the Journal of Machine Learning Research.

Secondly, progress in any scientific field relies to a huge extent on the

amount of available information. More available information equals more and

better science. Some of Kaelbling's colleagues, for example, want to design

robots that actually think like humans: Rodney Brooks, the director of

Kaelbling's laboratory, helped inspire Steven Spielberg's vision for the

movie "A.I." That enormously complicated task would be helped with input

from as many different scientists as possible. Putting everything online

seems like the obvious thing to do, Kaelbling argued.

Still, numerous scientists and publishers fear that moving away from

printed, subscription-based journals could derail standards and practices

that have worked well for years. Others fear that such a change could

shutter prestigious and important journals.

In the machine-learning community, swords have already crossed. In their

letter resigning from Machine Learning, the rebellious members, who

represented about two-thirds of the board, wrote: "Journals should

principally serve the needs of the intellectual community, in particular by

providing the immediate and universal access to journal articles that modern

technology supports, and doing so at a cost that excludes no one."

In the past, scholars in the field researched and wrote their articles,

submitted them to Machine Learning, and then waited up to a year to see them

in print. Probably most aggravating to the authors, Machine Learning's

owners, Netherlands-based Kluwer Academic Publishers, retained complete

Web pages - a fairly restrictive policy for the industry that the company

changed after the mass resignation.

With the Journal of Machine Learning Research, authors just e-mail their

pieces to Kaelbling, who then forwards them to assorted editors. These

volunteers, generally prestigious researchers in whatever particular

sub-field the article covers, then decide whether to accept or reject the

articles. If accepted, the articles appear online immediately and the

authors retain full copyrights.

"Mostly, it's just a bunch of work," said Kaelbling, before noting that she

still spends vastly more time with her MIT students and Erik the Red, a

robot resembling R2D2 that she is trying to teach to see and navigate

through hallways.

Despite Kaelbling's optimism, though, other scientists argue that there are

holes in her boat. Robert Holte, the editor of Machine Learning, supports

the new journal and suggests that both his journal and Kaelbling's can exist

harmoniously. But, he added, "What [the Journal of Machine Learning

Research] doesn't have right now is a history. You can have the most famous

people on your editorial board that you like. But until a journal has a

well-established track record proving its ability to attract a large number

of high quality, highly-cited papers, it cannot claim to be the community's

flagship journal."

Tenure committees, for example, know that being published in Machine

Learning means you've accomplished something significant. Until it earns a

reputation, Kaelbling's journal could just be two crackpots in a barn with a

cable modem.

In addition, advocates of print journals argue that online journals may not

hold to the same quality standards, becoming, in a sense, the scientific

equivalents of the Drudge Report.

"Paper journals have a strict limit on the number of papers they can

publish. With an online journal, there's always a temptation to accept

rather than reject," said Cornell professor Shai Ben-David, one member of

the editorial board of Machine Learning who chose not to resign.

Kaelbling acknowledged that her journal hasn't earned prestige yet, but she

insists that it will maintain strict standards. "It's the same people doing

the same work," she said, noting that the editorial board of the Journal of

Machine Learning Research is made up of many people who used to work for

Machine Learning. Kaelbling is one of the most highly respected scientists

in her field and, before resigning last year, she herself reviewed papers

for her print rival.

Kaelbling faces another potential problem in that no major for-profit

publisher supports and promotes her journal. Kluwer Academic Publishing, the

world's second largest scientific publisher with 731 journals under its

umbrella, stated that it supports Machine Learning by providing services

that include promotion, copy-editing, distribution and representation of the

journal at conferences.

Still, the nonprofit MIT Press does offer Kaelbling's journal its support,

publishing and promoting quarterly bound editions of the articles that have

appeared on the journal's Web site. "I don't think they are any less

effective than Kluwer," Kaelbling said.

MIT Press does publish and promote quarterly bound editions of the articles

that have appeared on Kaelbling's Web site, but has garnered fewer than 100

subscriptions so far. That doesn't phase Kaelbling, either. "Everyone's

going to have to do this," she said.

Even some scientists closely attached to old print publications agree.

According to Thomas Dietterich, a former editor of Machine Learning, and one

of the recent defectors to the new journal: "I am emotionally attached to

Machine Learning. I have every issue from the start to now. But things

change. In the computer business, we are used to technology turning things

upside down."