Can't upload citations due to duplicate file import error

Support
#1

I am trying to upload a RIS or BIB file of citations but Paperpile is not accepting this group of references:
“Duplicate File Import
These files were not imported because they appear to be identical to a file already in your library.”

I am doing a systematic review and need to categorise my papers according to the database they were found in. Even if every single reference in my file is a duplicate, I want to create this folder of papers.

I have tried uploading the citations in RIS and in BIB formats, renaming the upload file etc., and deleting other folders and their content (to hopefully remove some of the duplicates) to allow me to upload this collection - these things have not worked. I have also added a random citation into the collection that is definitely not in my library to ensure that a file is unique, but Paperpile is still not accepting it. I can’t see any logic to why this collection of references is not being accepted.

#2

I managed to add the folder to Paperpile by setting up a new Mendeley account online, uploading the folder to Mendeley (containing 263 items), and then importing into Paperpile. In Paperpile, there were only 262 items. This is not because one item was duplicated, because many more than one items are duplicates.

After exporting citations to csv and comparing with refworks exports, I wonder if this problem was caused by a paper titled:
** Economic and Social Impact of Increasing Uptake of Cardiac Rehabilitation Services–A Cost Benefit Analysis**

This is a duplicate paper. I have another one in the same file (of 263 citations) with a title containing only standard letters.

I can’t tell for sure unfortunately, but that is my best guess as to what the missing paper is - and therefore what the problem with uploading the RIS / BIB file was. This title with unusual characters is the only difference between my refworks library and the Paperpile folder when I export both and compare.

The interesting thing is that I can’t find the paper with unusual characters (pasted above) in my refworks library online, although it is there when I export to excel. Although reworks online says that it has 263 references, my refworks library only shows the title without special characters when I search for keywords/authors.

I feel satisfied that I haven’t lost a useful paper so am able to move on with my review. I have no idea if any of this is relevant to share but maybe it counts as a bug I experienced now rather than a support request.

Thanks

#3

We currently prevent imports if >90% of the entries are already in your library. We do this to avoid all the complications that come along with filling a library with duplicates. Duplicate-detection is imperfect so repeated importing of the same library leads to a frustrating and confusing user experience.

We realize this is imperfect for people doing systematic reviews and we will have to think more about handling this case. Currently if you really do wish to import the same papers again and deal with the troubles this results in, you can technically pad the import file with fake entries and the import should proceed as normal. This may be easier than the Mendeley workaround you found.

#4

Thanks Jason,
I didn’t realize the threshold was at 90%, which explains why adding only one extra citation didn’t do the trick!