Best way to import from Bookends?

I have a large Bookends library with 3,000+ references and PDFs that I’d like to import into Paperpile and essentially leave Bookends behind. What would be the best way to get this library exported and - importantly - have all the PDFs attaching correctly to the references in Paperpile as well. All of this without doing a ‘PDF download’ operation on imported references or importing based on the PDFs as opposed to the references themselves (which works reasonable well overall, but isn’t an option for a decade old library with a large number of PDFs attached).

Thanks for a great product!

I have been trying to play around with the Bookends exporter since you could export to multiple file formats, including RIS. However, standard RIS does not include information about attachment, so I have to add that manually. Here’s an example:

TY - JOUR
AU - Gandon, S
AU - Day, T
AU - Metcalf, CJ
AU - Grenfell, BT
T1 - Forecasting Epidemiological and Evolutionary Dynamics of Infectious Diseases.

PY - 2016 Oct
JA - Trends Ecol Evol
SP - 776-788
VL - 31
IS - 10
SN - 0169-5347
AB - Mathematical models have been powerful tools in developing mechanistic understanding of infectious diseases. Furthermore, they have allowed detailed forecasting of epidemiological phenomena such as outbreak size, which is of considerable public-health relevance. The short generation time of pathogens and the strong selection they are subjected to (by host immunity, vaccines, chemotherapy, etc.) mean that evolution is also a key driver of infectious disease dynamics. Accurate forecasting of pathogen dynamics therefore calls for the integration of epidemiological and evolutionary processes, yet this integration remains relatively rare. We review previous attempts to model and predict infectious disease dynamics with or without evolution and discuss major challenges facing the development of the emerging science of epidemic forecasting.

UR - http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=27567404
L1 - file:///Users/andersen/Dropbox%20(ALab)/Lab/Publishing/Library/Forecasting%20Epidemiological%20and%20Evolutionary%20Dynamics%20of%20Infectious%20Diseases.pdf
ER -

I included the ‘L1’ tag to include the PDF file, but when I then follow the Endnote import instructions (select exported .ris file as well as the folder with the PDFs), then the references is added twice - once from the PDF (I assume) and one from the RIS.

Is Paperpile expecting a very specific RIS format so it doesn’t see my ‘L1’ tag? Is there a way to fix this?

Alright, so I’m continuing my quest here and I’m getting closer. I tried using BibTeX as an export format from Bookends and now it seems like I can get attachments, DOIs, and PMIDs. HOWEVER, whenever I import from BibTeX all articles are classified as ‘miscellaneous’ instead of Journal article. If I export as RIS the articles are correctly classified, but then I don’t get PMIDs or DOIs, leaving the references incomplete.

A couple of critical problems with the ‘miscellaneous’ classification: (1) There is no way to reclassify multiple references, and (2) when the references are classified as ‘miscellaneous’ Paperpile can’t match the article up with online resources so you can’t download PDFs, update metadata, etc.

This is what my BibTeX format looks like for the same article as above:

@Article{,
title = {Forecasting Epidemiological and Evolutionary Dynamics of Infectious Diseases.},
author = {Gandon, S and Day, T and Metcalf, CJ and Grenfell, BT},
abstract = {Mathematical models have been powerful tools in developing mechanistic understanding of infectious diseases. Furthermore, they have allowed detailed forecasting of epidemiological phenomena such as outbreak size, which is of considerable public-health relevance. The short generation time of pathogens and the strong selection they are subjected to (by host immunity, vaccines, chemotherapy, etc.) mean that evolution is also a key driver of infectious disease dynamics. Accurate forecasting of pathogen dynamics therefore calls for the integration of epidemiological and evolutionary processes, yet this integration remains relatively rare. We review previous attempts to model and predict infectious disease dynamics with or without evolution and discuss major challenges facing the development of the emerging science of epidemic forecasting.},
type = {Journal article},
journal = {Trends Ecol Evol},
volume = {31},
number = {10},
pages = {776–788},
year = {2016},
url = {http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=27567404},
file = {file:///Users/andersen/Dropbox%20(ALab)/Lab/Publishing/Library/2016%20-%20Gandon%20-%20Forecasting%20Epidemiological%20and%20Evolutionary%20Dynamics%20of%20Infectious%20Diseases.pdf},
issn = {0169-5347},
pmid = {27567404},
doi = {10.1016/j.tree.2016.07.010}
}

I output type in this particular export, but if I leave that out, the article is also misclassified.

Any idea how to fix this?

Screenshots:
BibTeX:

RIS:

Taking a look at the BibTeX entry you posted, it seems that it is not valid BibTeX. The citation key is missing.
If you change it to “@Article{somekey,” it should imported correctly.

Hi Andreas,

Yup, put in an extra field and now it’s importing correctly. One down, but now I ran into a separate problem and it has to do with the attachments. A couple of things:

  1. I created an export file containing three references with three attached pdfs.
  2. Two of the references have pdfs with DOIs, etc., the third reference does not.
  3. For important I drag the BibTeX file and three pdfs to the window - the importer says ‘4 files’
  4. The import finishes saying ‘uploaded 4 paper’.

The last bit here is the problem - it’s actually three papers, not four. How does the importer handle the pdfs exactly? It seems to me that it’ll import the references and pdfs separately and then if the pdfs contain metadata it’ll merge them. It seems like it isn’t taking into consideration the ‘file’ part of the BibTeX import?

Here’s a screenshot - the problematic reference is the ‘Billingham’ paper where you can see both a (failed) pdf import and the imported BibTeX reference (the other two references are correctly imported - the reason why the pdfs aren’t attached in this instance is because those references already existed in my library and had pdfs attached)

Is there a way to make sure that Paperpile will actually attach the ‘file’ portion of the BibTeX and not import the pdfs separately?

Can you, please, send the BibTeX file to andreas@paperpile.com. I can then have a look and see what is going wrong.

For the “file” field we follow the structure proposed by JabRef, which is:
file = “some name for the pdf file:path/to/the/file.pdf:PDF”

In your case it would be:
file = {:file:///Users/andersen/Dropbox%20(ALab)/Lab/Publishing/Library/2016%20-%20Gandon%20-%20Forecasting%20Epidemiological%20and%20Evolutionary%20Dynamics%20of%20Infectious%20Diseases.pdf:PDF}

We also located a bug in our code. The %20 signs are not correctly translated to white space. We will fix it with the next release. In the meantime a search and replace for “%20” to " " should do the trick.

Nope, doesn’t work I’m afraid. I even tried removing all spaces from filenames. This is what I tried to import:

@Article{1953,
title = {Actively Acquired Tolerance of Foreign Cells.},
author = {Billingham, RE and Brent, L and Medawar, PB},
abstract = {},
type = {Journal article},
journal = {Nature},
volume = {172},
number = {4379},
pages = {603–606},
year = {1953},
url = {http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=13099277},
file = {file:///Users/andersen/Desktop/1953-Billingham.pdf},
issn = {0028-0836},
pmid = {13099277},
doi = {}
}

I tried changing

file = {file:///Users/andersen/Desktop/1953-Billingham.pdf},

to

file = {/Users/andersen/Desktop/1953-Billingham.pdf},

and iterations to that - no dice.

This should work:

file = {:file:///Users/andersen/Desktop/1953-Billingham.pdf:PDF}

We will make it easier with the next release. Then

file = {file:///Users/andersen/Desktop/1953-Billingham.pdf}

should work, too.

What is the current syntax? Neither of the described options seems to work. So weird that this is not documented. Thanks