It's a well-kown fact, that e-commerce development can face many unforseeable difficulties, which should be dealt with in no time. Some people are pushed away by them, but as for us, complicated situations help us to find non-standard solutions and become better this way. Not long ago one of our developers faced quite a challenging task of transfering a store catalogue based on tiu.ru platform to CS-Cart. All we had was only a csv-file with various products data, but it hadn't all the neccessary info for transfer. We thought, that this file would have SEO-links of products, and with their help we could get page's code and parse it. But our developer had to do everything differently.
Usage of a script
Detailed examination of products' URLs showed us, that they were formed on the basis of unique ID and a category code. Store platform performed redirecting well, even when category's name didn't match fully, but IDs did. The script, written by our developer, gathered info from pages, based on IDs, but at 200 pages it was blocked, due to available amount of requests violation. Proxy couldn't help either, due to https connection. Then the script was modified to pause for 30 seconds after every 30 requests. This way our developer was able to gather info on approximately 500 products. After deleting duplicates and mismatches, the catalogue was optimized to 250 products, which were then transferred to CS-Cart.