Start a new topic

Replace and adding words - Portia / Scrapy

Hi dears, just 2 quick doubts: 


1- I want my final JSON file to replace the text extract (for example text extracted is ADD TO CART but I want to change to IN STOCK in my final JSON. Is it possible?


2- I also would like to add some custom data to my final JSON file that is not in the website, for example "Store name"... so every product that I scrape will have the store name after it. Is it possible?


I am using both Portia and Scrapy so your suggestions are welcome in both platforms.


Just as an example my Scrapy code is partially copied below:

class Advent(BasePortiaSpider):

    name = "advent"

    allowed_domains = [u'www.adventgames.com.au']

    start_urls = [u'http://www.adventgames.com.au/c/4504822/1/all-games-a---k.html',

                  {u'url': u'http://www.adventgames.com.au/Listing/Category/?categoryId=4504822&page=[1-5]',

                   u'fragments': [{u'valid': True,

                                   u'type': u'fixed',

                                   u'value': u'http://www.adventgames.com.au/Listing/Category/?categoryId=4504822&page='},

                                  {u'valid': True,

                                   u'type': u'range',

                                   u'value': u'1-5'}],

                   u'type': u'generated'}]

    rules = [

        Rule(

            LinkExtractor(

                allow=('.*'),

                deny=()

            ),

            callback='parse_item',

            follow=True

        )

    ]

    items = [

        [

            Item(

                PortiaItem,

                None,

                u'.DataViewCell > form > table',

                [

                    Field(

                        u'Title',

                        'tr:nth-child(1) > td > .DataViewItemProductTitle > a *::text',

Login to post a comment