arXiv Index: 2 GB corpus of biomedical open research indexed for text analysis / data mining

Tuesday, May 1, 2012

2 GB corpus of biomedical open research indexed for text analysis / data mining

As of 01 May 2012 BioMed Central (with Chemistry Central and SpringerOpen) has published 123400 articles of peer-reviewed research, all of which are covered by our open access license agreement which allows free distribution and re-use of the full-text article, including the highly structured XML version.
As a result, BioMed Central's open access corpus is ideally suited for use by text mining researchers.

http://www.biomedcentral.com/about/datamining

A priori

Cross-disciplinary reviews aimed at a wide audience.

Additionally there is an enormous amount of overlap in the application of math, lagrangian equations, and machine learning to physics, biology, linguistics and other fields.

Other sites covering arXiv include Technology Review's arXiv blog but they only post a select few chosen articles after a large portion of commentary. This blog intends on posting a much larger volume of research at a faster pace, with some explanations intended for a general audience but also plenty of depth, range, and volume of articles for genuine students, researchers, and those with proficiency and interest in science in a number of fields.

We'll try to avoid overly technical or math/jargon filled papers in lieu of those that would appeal to a wider audience, but we'll not shy away from interesting or thought-provoking important connections between multiple fields (such as the Goldbach / Riemann / prime number / renormalization connection)

We'll cover some news articles as well when the full text is not available or the story is sufficiently appropriate / connected to other stories to merit it. arXiv is merely the largest repository of free full access papers in existence, but the scope of our posts will include any other open access journal articles. arXiv is just the most recognisable of such journals.

Besides technical reviews and breakthroughs we also intend to index important articles in the social sciences should they be sufficiently relevant or interesting in and of themselves.