This is a web crawler which can download all the papers on Sci Rob in 5 mins.
The technique architecture are based on Scrapy
and Python
.
In order to use this crawler, you should make sure you have the access to the paper resources on science robotics website (right IP).
Finally, a pdf combination script (pdfcom.py) is provided for combining the papers belongs to one issue into one single pdf, also a pdf file containing all the downloaded pdfs are generated. Papers in the combined pdf file are arranged in their original order on the web.
A detailed tutorial of scrapy and this demo are given at here.