Export Attachments with RIS


#1

I’m coming over to PaperPile from Zotero. One of the benefits of Zotero is that it’s easy to get my references and attachments in and out of my database. For example, I just imported 2,000 odd references with attachments from Zotero into PaperPile with no issues.

Unless I’m missing something it seems more difficult to get my data out of PaperPile.

I can export all my references as an RIS file but the attachments are not present. In contrast, when I manually export from Zotero I can an RIS folder with all the references and attachments, which can then be imported into another database.

I’m concerned about this lock in with Paperpile, as I don’t really want to have to manually attach 1000s of PDFs to references should I decide in the future I don’t want to use PaperPile (or Paperpile goes out of business).

It there a way to export my Paperpile references with attachments into other bibliographic software?


#3

Please use the JSON export.

Paperpile is the opposite of lock in. Although it has turned out not to be a frequently asked questions we have made this the very first item in our FAQs when we started. https://paperpile.com/help


#4

Thank you for the reply. I think I must be missing something. When I export all my data as a JSON file I get a 3.2 mb .txt file. This has all my citations but not PDF attachments. My concern is over the attachments too. Although the attachments are of course available in Google Drive, how would I export all my bibliographic data with the PDF attached to each citation such that I can import it all into another program should Paperpile shut down or not meet my needs, in much the same way that I’ve just imported my entire Zotero database with attachments into Paperpile.


#5

Can endnote or other ref manager import json files??


#6

No, the JSON format is meant to provide you with a complete dump of your data in a machine readable format. To transfer your data to other reference managers please use RIS or BibTeX.

Please note that these formats are not standardized formats and things like attached files or folders/tags are not perfectly specified in these format. To provide a full-proof migration between all possible programs would require a special exporter for each product.


#7

Please have a look at my comment above. It’s impossible to provide an exporter for every product. The JSON includes the links to your files in Google Drive and Zotero has an API so there is no risk of lock-in.

Please understand that our focus is on our own product because as you can see this forum is full of suggestions from customers who want an even better Paperpile and not move away…


#8

My question is about your product. It’s about export from Paperpile. I too want the focus to be on a better product, and I hope not to move away. And a better product means having a good export. You gain customer trust this way.

However, Mendeley got bought out as did Papers, with Papers being pretty rubbish now. Endnote used to be the standard in academia, now younger researchers hardly use it. People are increasingly dissatisfied with Evernote. For an undergraduate the long-term usage of Paperpile may not matter. For academics it’s rather different. We need to be sure we can get our data out, and that’s what makes Paperpile even better: there’s no guarantee your product will be around in ten years time.

So my original question still stands, and it’s very much a question about your product. If you are offering RIS export anyway, and given that just about all other reference managers use RIS, can it not be implemented in such a way that the attachments are with the RIS file? This is what Zotero does so it must be possible. It just means that if you go bust or your product goes bad, or you have a buy out, then our data is readily available, both the bibliographic references and the PDF attachments.

The fact that I’m coming to Paperpile from Zotero, and deciding between you and Bookends for long-term use, as I’m sure many of your customers are, should indicate that many of your customers would like Paperpile to offer a good export option.

The alternative is to have to re-attach thousands of PDFs manually to each record and this very much is not a good export option.


#9

I don’t know all the technical details off the top of my head. Maybe @andreas knows more.

But from what I know how difficult it is to import from all the different RIS formats (which is supposed to be one format) I’m pretty sure it’s impossible to write one RIS file with attachment information that can be read by EndNote, Mendeley, Papers, Zotero, Bookends etc.

I know your question is about our product but any answer is meaningless without taking into consideration the other products.

We could write a RIS with some file information pointing to a Google Drive file resource. But no other product will be able to read it. So we would need to write a specific exporter for all the products and test them with all the versions on all operating systems. This is what we do for our importers and it’s not easy to get that right.

I hope you understand that (like all the other products) we cannot do that in the other direction.


#10

Firstly, I’m not familiar with RIS export at all. But I was wondering if a manual work-around is possible, pending @andreas 's feedback.

Here’s what I might try - (1) sync the paperpile pdf folder to a local folder, using Google Drive Sync on a windows system ; (2) export the desired ref as a RIS file ; (3) open the RIS file with a text editor ; (4) replace the portion of the paths in L1 fields with the sync paperpile location, using just simple text-based find-and-replace (For examples, \All Papers\ … replace with xxx\xxx\All Papers…) . I haven’t tried it, but just wondering theoretically if it might work.


#11

Sounds plausible. I will try it later this week. Thank you.

However, with Zotero for Firefox shutting down next month or so, which was my preferred database, and Zotero Standalone shifting platforms in 2017, it’s another reason demonstrating the necessity of being able to get data out.

Even if it’s not the intention of Paperpile’s creators the product effectively locks us in. We can indeed get references out. We can get the PDFs out. But we can’t get both out together attached to each other. With a database of thousands of references and attachments this makes things rather awkward if we ever need to take our data out as I’m finding I currently need to with Zotero.

I really want to use Paperpile and give you guys money, but until there’s a satisfactory export option I can’t commit. My database is too valuable for that.

Similarly, we want to use Paperpile for our gene tech company for all our staff, as we are moving from Zotero, because Paperpile is so good and user friendly, and it gells really good with G Suite that we use. But, for the reasons above, we need to be sure we can get our data and attachments out together in a satisfactory way. Reluctantly, we are going to have consider Mendeley.

I hope you will consider this in the future.


#12

To be clear about that. Our JSON does include all necessary information to link your references to the files on Google Drive.

There are two angles here:

  1. In this forum we discuss features with the idea of adding them to the main product and it has to compete with all other features. I can’t remember a single request for this in 3 years so priority is low to begin with. Second it has no easy solution because we are a cloud based system and the files are by default not available on your hard disk. So we can’t simply write a file path to a RIS file.

  2. Of course it’s possible to sync the files to your hard disk using the Google Drive client and then add the information from the JSON to the RIS in a format that can be read by Zotero. That’s a very specific problem which is easy to solve in about 20 lines of code. We do custom things like that for universities with site licenses and bigger companies to assist migration. We have done this before and could do this for your company. However, I still think the JSON is what you actually want because it contains the information about labels, folders, notes,… It’s impossible to migrate this rich data model to a different rich data model like Zotero or Mendeley if you force it through a RIS file which cannot represent this data properly.


#13

I checked our RIS export and the relative path to PDF attachments (as they are stored in Google Drive: All Papers/[A-Z]/filename.pdf) is correctly written to the L1 field in the RIS file. As Stefan wrote above in (1) we can’t just write the absolute path to the PDF file there, as we do not know to what exact location you are syncing your Google Drive content to.

But the problem can be solved by a simple find & replace in a text editor. Just replace
L1 - All

with the actual location on your hard drive, e.g.

L1 - file:///home/agruber/test/All

In the example I tired, the L1 line then looked as follows:

L1 - file:///home/agruber/test/All Papers/K/Köhn and Hüttelmaier 2016 - Non-coding RNAs, the cutting edge of histone messages.pdf

I imported the RIS file in Zotero and the PDF file was correctly attached to the record.


#14

Thanks @andreas

I was not aware that we already write the “L1” line with the relative path. So it’s not 20 lines of code but only 2 lines of code (or a search and replace) to get the desired RIS file.


#15

Thank you for solving this. I had a hard time exporting the articles to endnote because my senior advisors use it. Because paperpile renames the PDF, endnote cannot identify the pdf metadata and I had to manually edit the references. Hopefully this will help me with the exports.