After we started to use version 2 of Portia we are experiencing unwanted deduplication of similar items from crawl. Looking through the logs of these crawl reveals that these items does indeed have different values for at least one field in each item. As we see it these items are not duplicates, and should not be discarded.
As a note. All fields in the item are configured with the Vary-option enabled, and both Required-options disabled.
Crawl logs were read on the web interface from https://app.scrapinghub.com/p/110257/36/4/log
Customer support service by UserEcho