Export Attachments with RIS

Please use the JSON export.

Paperpile is the opposite of lock in. Although it has turned out not to be a frequently asked questions we have made this the very first item in our FAQs when we started. https://paperpile.com/help

Thank you for the reply. I think I must be missing something. When I export all my data as a JSON file I get a 3.2 mb .txt file. This has all my citations but not PDF attachments. My concern is over the attachments too. Although the attachments are of course available in Google Drive, how would I export all my bibliographic data with the PDF attached to each citation such that I can import it all into another program should Paperpile shut down or not meet my needs, in much the same way that I’ve just imported my entire Zotero database with attachments into Paperpile.

1 Like

Can endnote or other ref manager import json files??

No, the JSON format is meant to provide you with a complete dump of your data in a machine readable format. To transfer your data to other reference managers please use RIS or BibTeX.

Please note that these formats are not standardized formats and things like attached files or folders/tags are not perfectly specified in these format. To provide a full-proof migration between all possible programs would require a special exporter for each product.

Please have a look at my comment above. It’s impossible to provide an exporter for every product. The JSON includes the links to your files in Google Drive and Zotero has an API so there is no risk of lock-in.

Please understand that our focus is on our own product because as you can see this forum is full of suggestions from customers who want an even better Paperpile and not move away…

My question is about your product. It’s about export from Paperpile. I too want the focus to be on a better product, and I hope not to move away. And a better product means having a good export. You gain customer trust this way.

However, Mendeley got bought out as did Papers, with Papers being pretty rubbish now. Endnote used to be the standard in academia, now younger researchers hardly use it. People are increasingly dissatisfied with Evernote. For an undergraduate the long-term usage of Paperpile may not matter. For academics it’s rather different. We need to be sure we can get our data out, and that’s what makes Paperpile even better: there’s no guarantee your product will be around in ten years time.

So my original question still stands, and it’s very much a question about your product. If you are offering RIS export anyway, and given that just about all other reference managers use RIS, can it not be implemented in such a way that the attachments are with the RIS file? This is what Zotero does so it must be possible. It just means that if you go bust or your product goes bad, or you have a buy out, then our data is readily available, both the bibliographic references and the PDF attachments.

The fact that I’m coming to Paperpile from Zotero, and deciding between you and Bookends for long-term use, as I’m sure many of your customers are, should indicate that many of your customers would like Paperpile to offer a good export option.

The alternative is to have to re-attach thousands of PDFs manually to each record and this very much is not a good export option.

1 Like

I don’t know all the technical details off the top of my head. Maybe @andreas knows more.

But from what I know how difficult it is to import from all the different RIS formats (which is supposed to be one format) I’m pretty sure it’s impossible to write one RIS file with attachment information that can be read by EndNote, Mendeley, Papers, Zotero, Bookends etc.

I know your question is about our product but any answer is meaningless without taking into consideration the other products.

We could write a RIS with some file information pointing to a Google Drive file resource. But no other product will be able to read it. So we would need to write a specific exporter for all the products and test them with all the versions on all operating systems. This is what we do for our importers and it’s not easy to get that right.

I hope you understand that (like all the other products) we cannot do that in the other direction.

Firstly, I’m not familiar with RIS export at all. But I was wondering if a manual work-around is possible, pending @andreas 's feedback.

Here’s what I might try - (1) sync the paperpile pdf folder to a local folder, using Google Drive Sync on a windows system ; (2) export the desired ref as a RIS file ; (3) open the RIS file with a text editor ; (4) replace the portion of the paths in L1 fields with the sync paperpile location, using just simple text-based find-and-replace (For examples, \All Papers\ … replace with xxx\xxx\All Papers…) . I haven’t tried it, but just wondering theoretically if it might work.

Sounds plausible. I will try it later this week. Thank you.

However, with Zotero for Firefox shutting down next month or so, which was my preferred database, and Zotero Standalone shifting platforms in 2017, it’s another reason demonstrating the necessity of being able to get data out.

Even if it’s not the intention of Paperpile’s creators the product effectively locks us in. We can indeed get references out. We can get the PDFs out. But we can’t get both out together attached to each other. With a database of thousands of references and attachments this makes things rather awkward if we ever need to take our data out as I’m finding I currently need to with Zotero.

I really want to use Paperpile and give you guys money, but until there’s a satisfactory export option I can’t commit. My database is too valuable for that.

Similarly, we want to use Paperpile for our gene tech company for all our staff, as we are moving from Zotero, because Paperpile is so good and user friendly, and it gells really good with G Suite that we use. But, for the reasons above, we need to be sure we can get our data and attachments out together in a satisfactory way. Reluctantly, we are going to have consider Mendeley.

I hope you will consider this in the future.

To be clear about that. Our JSON does include all necessary information to link your references to the files on Google Drive.

There are two angles here:

  1. In this forum we discuss features with the idea of adding them to the main product and it has to compete with all other features. I can’t remember a single request for this in 3 years so priority is low to begin with. Second it has no easy solution because we are a cloud based system and the files are by default not available on your hard disk. So we can’t simply write a file path to a RIS file.

  2. Of course it’s possible to sync the files to your hard disk using the Google Drive client and then add the information from the JSON to the RIS in a format that can be read by Zotero. That’s a very specific problem which is easy to solve in about 20 lines of code. We do custom things like that for universities with site licenses and bigger companies to assist migration. We have done this before and could do this for your company. However, I still think the JSON is what you actually want because it contains the information about labels, folders, notes,… It’s impossible to migrate this rich data model to a different rich data model like Zotero or Mendeley if you force it through a RIS file which cannot represent this data properly.

I checked our RIS export and the relative path to PDF attachments (as they are stored in Google Drive: All Papers/[A-Z]/filename.pdf) is correctly written to the L1 field in the RIS file. As Stefan wrote above in (1) we can’t just write the absolute path to the PDF file there, as we do not know to what exact location you are syncing your Google Drive content to.

But the problem can be solved by a simple find & replace in a text editor. Just replace
L1 - All

with the actual location on your hard drive, e.g.

L1 - file:///home/agruber/test/All

In the example I tired, the L1 line then looked as follows:

L1 - file:///home/agruber/test/All Papers/K/Köhn and Hüttelmaier 2016 - Non-coding RNAs, the cutting edge of histone messages.pdf

I imported the RIS file in Zotero and the PDF file was correctly attached to the record.

3 Likes

Thanks @andreas

I was not aware that we already write the “L1” line with the relative path. So it’s not 20 lines of code but only 2 lines of code (or a search and replace) to get the desired RIS file.

Thank you for solving this. I had a hard time exporting the articles to endnote because my senior advisors use it. Because paperpile renames the PDF, endnote cannot identify the pdf metadata and I had to manually edit the references. Hopefully this will help me with the exports.

For me, the most convenient way turned out to be to export the RIS file into the Google Drive/Paperpile folder and then import into Zotero from there. This way, the relative paths to the PDF files are correct and no editing of the RIS files is needed.

Unfortunately this doesn’t work correctly if a file is in more than one folder. You get a mess like this:

L1 - All Papers/TWLA Reading Group/Wishlist,All Papers/Teaching/Leiden Course S02,All Papers/Teaching/Leiden Course S02/FInal Selections/Silvio 2019 - Puppets, Gods, and Brands - Theorizing the Age of Animation from Taiwan.pdf

Other software can’t read this. I don’t know what the best solution would be, but some options might be to (a) only list the first path to the document (since other apps generally don’t need the other info), (b) list each path on a separate line, or with clear markers between them (i.e. don’t do this: “folder 1, folder 2” in the middle of a path, and don’t use symbols that might be in book titles like commas to separate entries), or ( c) export the PDF with the RIS file in its own attachment folder (this is what Zotero does).

Thanks for pointing this out. It is a bug and we will fix it with the next release.

1 Like

Has there been a new release yet? I just checked and I’m still seeing folder paths like “All Papers/Starred Items,Starred Papers/…” which might make sense to Paperpile, but won’t make sense to other apps…

OK. This seems to be working now. To get Zotero to link to the files in the paperpile folder on Google I had to use a different naming/filing system however. This is configurable in the preferences. The most important was to have the PDFs not use spaces in the names.

Hi @Kerim,

Could you comment in more detail as to how you integrate zotero with paperpile? I’m using obsidian for most of my writing these days and I am pretty entrenched in paperpile, but there is an active community developing links between obsidian and zotero and it may help to have a setup that can leverage that effort.

Thanks
H

Similar situation here, although I use Logseq, not Obsidian. I still prefer Paperpile to Zotero, even with the new version 6, but cross-app integration is key.

  1. Do a RIS export from Paperpile
  2. Move that file to the top level of your Google Drive Paperpile folder
  3. Import in Zotero, linking to the existing files rather than copying them to Zotero

Note, however, that this might fail if you have spaces in your PDF file names. To get it to work I had to first click on the drive menu on the top-right of the paperpile screen and choose “configure”. This is my configuration for naming/organizing PDFs: [year]/[firstauthor]-[year]-[titleshort]

There is a plugin for Obsidian (sorry, I forget what it is called) that can read Paperpile’s BibLaTeX export file which is perpetually updated, this might be a good option for you, but it doesn’t exist for Logseq.

2 Likes