initBasti / Amazon2PlentySync (public) (License: GPLv3) (since 2019-01-27) (hash sha1)
Transfer your data from you Amazon Flatfile spreadsheet over to the Plentymarkets system. How to is included in the readme
Clone URLs: https://rocketgit.com/user/initBasti/Amazon2PlentySync ssh://rocketgit@ssh.rocketgit.com/user/initBasti/Amazon2PlentySync git://git.rocketgit.com/user/initBasti/Amazon2PlentySync
automatic_internnumber_handling category-config develop ebay-properties elastic-version feature/amazon-data feature/graphical-user-interface fix_new_version_config_bug home-products master release/v0.1-CLI remove-fixedvalues
List of commits:
Subject Hash Author Date (UTC)
Remove manual upload of extern id & export files 66d60bc9243173f256b01071f203849edf43546d Sebastian Fricke 2020-04-28 13:55:27
add tests folder pycache to gitignore 9d7989d1658d3dffd7eb00e9411af3dec6c85563 Sebastian Fricke 2020-04-27 12:55:24
Unit test for find_price function c19fa28ad9bf0dd1b5361e871dc498f6704cb196 Sebastian Fricke 2020-04-27 12:53:20
Remove price calculation from script d852cb3ef648336ed94daeaaa220721e6055dd7c Sebastian Fricke 2020-04-27 12:50:52
Function description priceUpload 2c5735f4f4c79607f55c77b1359aa5984d1e6699 Sebastian Fricke 2020-04-23 09:34:30
similar attribute search regardless of the parent 4bb0970408d78d25264d479428fe8c3389483215 Sebastian Fricke 2020-04-23 09:28:19
Fix marking dropdown bug 0d1695083d2b4e49fd364a36a7ef3c92a192d62f Sebastian Fricke 2020-04-23 09:26:46
update desktop specific build script for windows 8a6536d6173f7383022fab92f234ab25fc81204b Sebastian Fricke 2020-04-22 10:15:05
Refactor config handling a9be950a4e8d97fa97c7e9caff33fcbb1f476d9d Sebastian Fricke 2020-04-22 10:11:57
Fix gui/category_chooser: file change bug 9879e65c9aad9b1feb05b6121a0e33c129a8beb5 Sebastian Fricke 2020-04-22 10:09:36
update .gitignore 6c7628af605a72ced1e146c27da8639225ab9c6c Sebastian Fricke 2020-04-22 10:08:47
Fix error.py: Fix colorful messages on windows 31f0b106c7aee1962bba3efb5758f647170eceff Sebastian Fricke 2020-04-22 10:07:30
Enhanced error output to extra module dda37a22d21db1af6330fd62395682b91f46f5ec Sebastian Fricke 2020-01-27 10:56:00
Introduction infoPrint, removed unnecessary parameter 98b6779f95fcf6d3350f64e7d2a8151932415458 Sebastian Fricke 2020-01-16 14:27:15
Add Cdiscount price & coding style review 7c5451b067760100904aed947b2ba484ab9c9e45 Sebastian Fricke 2020-01-16 14:25:15
coding style, naming conventions & uninitialized 218378ca191510dc2092108a87e63ff83b4c6b31 Sebastian Fricke 2020-01-16 13:58:24
coding style & cleanup image upload module 04ac13fb764baecd1a419cbffcd9696a3ff5b680 Sebastian Fricke 2020-01-16 13:56:07
coding style improvements and colorful error msg 401db811c0edd62b14c7a0a29e2c1f6d2791774c Sebastian Fricke 2020-01-16 13:54:25
code cleanup and coding style category.py b1e41b45fe405d3826a9b6e452fb9e2f9f6697bf Sebastian Fricke 2020-01-16 10:43:44
Added the category config to the gitignore file b8b522d9ade4b59b5d0a0bd4f9b7c79e8db334c6 Sebastian Fricke 2020-01-15 14:56:06
Commit 66d60bc9243173f256b01071f203849edf43546d - Remove manual upload of extern id & export files
Adjust the config to take two additional fields for a extern ID file and
the URL of the plentymarkets Export required for the Variation ID.
Refactor the get_variation_id function to work with the pandas library

Create function get_externalid, which retrieves the external id stored
within a local file for each sku.

Add config handling in product_import for the new fields.
Author: Sebastian Fricke
Author date (UTC): 2020-04-28 13:55
Committer name: Sebastian Fricke
Committer date (UTC): 2020-04-28 13:55
Parent(s): 9d7989d1658d3dffd7eb00e9411af3dec6c85563
Signing key:
Tree: 8ae445ae96f06fd3afb521a0daf129e5c93d1127
File Lines added Lines deleted
packages/config.py 3 1
packages/image_upload.py 54 60
packages/item_upload.py 92 45
product_import.py 27 22
File packages/config.py changed (mode: 100644) (index 2610a38..03ff003)
... ... def createConfig(name):
94 94 'upload_folder':upload_folder, 'upload_folder':upload_folder,
95 95 'data_folder':input_folder, 'data_folder':input_folder,
96 96 'attribute_file':'', 'attribute_file':'',
97 'file_change_date': ''
97 'file_change_date': '',
98 'internnumbers': '',
99 'export_plentymarkets': ''
98 100 } }
99 101 config['CATEGORY'] = { config['CATEGORY'] = {
100 102 'example-category':'category ID (integer)' 'example-category':'category ID (integer)'
File packages/image_upload.py changed (mode: 100644) (index 8cba657..da965c0)
... ... def getColorAttributeID(attributefile, product):
36 36
37 37 def imageUpload(flatfile, attributefile, exportfile, uploadfolder, filename): def imageUpload(flatfile, attributefile, exportfile, uploadfolder, filename):
38 38
39 try:
40 data = SortedDict()
41
42 column_names = ['VariationID', 'Multi-URL', 'connect-variation', 'mandant',
43 'listing', 'connect-color']
44 attribute_id = ''
45 variation_id = 0
46
47 with open(flatfile['path'], mode='r', encoding=flatfile['encoding']) as item:
48 reader = csv.DictReader(item, delimiter=';')
49 for index, row in enumerate(reader):
50 linkstring = ''
51 imglinks = [
52 row['main_image_url'],
53 row['other_image_url1'],
54 row['other_image_url2'],
55 row['other_image_url3'],
56 row['other_image_url4'],
57 row['other_image_url5'],
58 row['other_image_url6'],
59 row['other_image_url7'],
60 row['other_image_url8']
61 ]
62
63 num = 1
64 variation_id = item_upload.getVariationId(
65 exportfile=exportfile, sku=row['item_sku'])
66 try:
67 if not imglinks[0]:
68 break
69 for img in [i for i in imglinks if i]:
70 if not searchSpecialImage(img):
71 if not linkstring:
72 linkstring += f"{img};{str(num)}"
73 num += 1
74 continue
75 linkstring += f",{img};{str(num)}"
76 num += 1
77 continue
78 print(f"\n{img} is a special image\n")
39 data = SortedDict()
40
41 column_names = ['VariationID', 'Multi-URL', 'connect-variation', 'mandant',
42 'listing', 'connect-color']
43 attribute_id = ''
44 variation_id = 0
45
46 with open(flatfile['path'], mode='r', encoding=flatfile['encoding']) as item:
47 reader = csv.DictReader(item, delimiter=';')
48 for index, row in enumerate(reader):
49 linkstring = ''
50 imglinks = [
51 row['main_image_url'],
52 row['other_image_url1'],
53 row['other_image_url2'],
54 row['other_image_url3'],
55 row['other_image_url4'],
56 row['other_image_url5'],
57 row['other_image_url6'],
58 row['other_image_url7'],
59 row['other_image_url8']
60 ]
61
62 num = 1
63 variation_id = item_upload.get_variation_id(
64 exportfile=exportfile, sku=row['item_sku'])
65 try:
66 if not imglinks[0]:
67 break
68 for img in [i for i in imglinks if i]:
69 if not searchSpecialImage(img):
79 70 if not linkstring: if not linkstring:
80 71 linkstring += f"{img};{str(num)}" linkstring += f"{img};{str(num)}"
81 72 num += 1 num += 1
82 73 continue continue
83 74 linkstring += f",{img};{str(num)}" linkstring += f",{img};{str(num)}"
84 75 num += 1 num += 1
76 continue
77 print(f"\n{img} is a special image\n")
78 if not linkstring:
79 linkstring += f"{img};{str(num)}"
80 num += 1
81 continue
82 linkstring += f",{img};{str(num)}"
83 num += 1
85 84
86 85
87 except Exception as err:
88 error.errorPrint(msg="Link string building failed",
89 err=err,
90 linenumber=sys.exc_info()[2].tb_lineno)
86 except Exception as err:
87 error.errorPrint(msg="Link string building failed",
88 err=err,
89 linenumber=sys.exc_info()[2].tb_lineno)
91 90
92 try:
93 attribute_id = getColorAttributeID(
94 attributefile=attributefile, product=row)
95 except Exception as err:
96 error.warnPrint(
97 msg=f"get attr ID of color {row['color_name']} failed",
98 linenumber=inspect.currentframe().f_back.f_lineno,
99 err=err)
91 try:
92 attribute_id = getColorAttributeID(
93 attributefile=attributefile, product=row)
94 except Exception as err:
95 error.warnPrint(
96 msg=f"get attr ID of color {row['color_name']} failed",
97 linenumber=inspect.currentframe().f_back.f_lineno,
98 err=err)
100 99
101 100
102 values = [variation_id, linkstring, 1, -1,
103 -1, attribute_id]
101 values = [variation_id, linkstring, 1, -1,
102 -1, attribute_id]
104 103
105 data[row['item_sku']] = dict(zip(column_names, values))
104 data[row['item_sku']] = dict(zip(column_names, values))
106 105
107 except Exception as err:
108 error.errorPrint(
109 msg=f"flatfile read failed on index:{index}",
110 err=err,
111 linenumber=sys.exc_info()[2].tb_lineno)
112 106
113 107 barcode.writeCSV(dataobject=data, name='Image_', columns=column_names, barcode.writeCSV(dataobject=data, name='Image_', columns=column_names,
114 108 upload_path=uploadfolder, item=filename) upload_path=uploadfolder, item=filename)
File packages/item_upload.py changed (mode: 100644) (index 581c241..175018e)
... ... import collections
5 5 import inspect import inspect
6 6 import chardet import chardet
7 7 import os import os
8 import pandas
9 import xlrd
8 10 from packages import barcode, amazon_data_upload, price, error from packages import barcode, amazon_data_upload, price, error
9 11
10 12
 
... ... def itemUpload(flatfile, intern, stocklist, folder, input_data, filename):
115 117 '', # externalID '', # externalID
116 118 '1', '1', # NetStock pos = Vis & neg = Invis '1', '1', # NetStock pos = Vis & neg = Invis
117 119 '2', input_data['categories'], '2', input_data['categories'],
118 input_data['categories'][0:3], input_data['categories'][0:3],
120 input_data['categories'][0:3],
121 input_data['categories'][0:3],
119 122 'Y', 'Y', # mandant 'Y', 'Y', # mandant
120 123 '', '', # barcode '', '', # barcode
121 124 'Y', 'Y', # marketconnection 'Y', 'Y', # marketconnection
 
... ... def itemUpload(flatfile, intern, stocklist, folder, input_data, filename):
137 140 except Exception as err: except Exception as err:
138 141 error.errorPrint("itemUpload: setting values failed", err, error.errorPrint("itemUpload: setting values failed", err,
139 142 sys.exc_info()[2].tb_lineno) sys.exc_info()[2].tb_lineno)
140 data[row['item_sku']] = collections.OrderedDict(zip(column_names, values))
143
144 data[row['item_sku']] =\
145 collections.OrderedDict(zip(column_names, values))
141 146 except KeyError as err: except KeyError as err:
142 147 error.errorPrint("Reading file failed", err, error.errorPrint("Reading file failed", err,
143 148 sys.exc_info()[2].tb_lineno) sys.exc_info()[2].tb_lineno)
144 149 return row['item_sku'] return row['item_sku']
145 150
146 # open the intern number csv to get the item ID
147 with open(intern['path'], mode='r', encoding=intern['encoding']) as item:
148 reader = csv.DictReader(item, delimiter=";")
149 for row in reader:
150 try:
151 if row['amazon_sku'] in list(data.keys()):
152 data[row['amazon_sku']]['ExternalID'] = row['full_number']
153 except KeyError as keyerr:
154 error.warnPrint("key was not found in intern number list",
155 sys.exc_info()[2].tb_lineno, keyerr)
151 # open the intern number xlsx to get the external id
152 get_externalid(dataset=data, numberlist=intern)
156 153
157 # Include the barcodes & asin
158 154 barcode_data = barcode.barcode_Upload(flatfile, stocklist) barcode_data = barcode.barcode_Upload(flatfile, stocklist)
159 155
160 156 for row in barcode_data: for row in barcode_data:
 
... ... def itemUpload(flatfile, intern, stocklist, folder, input_data, filename):
183 179 error.errorPrint("SKU part for "+row, err, error.errorPrint("SKU part for "+row, err,
184 180 sys.exc_info()[2].tb_lineno) sys.exc_info()[2].tb_lineno)
185 181
186 # Include the amazonsku
187 182 ama_data = amazon_data_upload.amazonDataUpload(flatfile) ama_data = amazon_data_upload.amazonDataUpload(flatfile)
188 183
189 184 for row in ama_data: for row in ama_data:
 
... ... def checkEncoding(file_dict):
470 465
471 466 return file_dict return file_dict
472 467
473 def getVariationId(exportfile, sku):
468 def get_variation_id(exportfile, sku):
469 """
470 Parameter:
471 exportfile [String] => Url of the plentymarkets export
472 from the config
473 sku [String] => Sku from the flatfile for matching
474 474
475 variationid = 0
476 with open(exportfile['path'], mode='r',
477 encoding=exportfile['encoding']) as item:
478 reader = csv.DictReader(item, delimiter=';')
475 Description:
476 Check if the export has the correct header and retrieve
477 the Variation ID of the matching SKU
479 478
480 for row in reader:
481 if 'VariationNo' in list(row.keys()):
482 if row['VariationNo'] == sku:
483 variationid = row['VariationId']
484 continue
485 try:
486 if row[list(row.keys())[1]] == sku:
487 for i in range(len(list(row.keys()))):
488 # matches .id .ID _ID _id ID id
489 if re.search(r'\bid', [*row][i].lower()):
490 print("found ID in {0} value: {1}"
491 .format(list(row.keys())[i],
492 row[list(row.keys())[i]]))
493 variationid = row[list(row.keys())[i]]
494 except Exception as err:
495 error.errorPrint("Looking for irregularities in getVariationId",
496 err, sys.exc_info()[2].tb_lineno)
497 if os.name == 'nt':
498 print("press ENTER to continue...")
499 input()
500 exit(1)
501 if not variationid:
502 error.warnPrint(msg="No Variation ID found for "+sku,
503 linenumber=inspect.currentframe().f_back.f_lineno)
504
505 return variationid
479 Return:
480 [String] => The variation number
481 0 => Failed to retrieve value
482 """
483 exp = pandas.read_csv(exportfile,
484 sep=';')
485
486 if not len(exp.index):
487 error.warnPrint(
488 msg='exp is empty, skip variation ID', err='',
489 linenumber=inspect.currentframe().f_back.f_lineno)
490 return 0
491
492 if(not len(exp.columns[exp.columns.str.contains(pat='Variation.id')]) or
493 not len(exp.columns[exp.columns.str.contains(pat='Variation.number')])):
494 error.warnPrint(
495 msg="Exportfile requires fields 'Variation.id'&'Variation.number'",
496 err='', linenumber=inspect.currentframe().f_back.f_lineno)
497 return 0
498
499 variation = exp[exp['Variation.number'] == sku]
500 if not len(variation.index):
501 error.warnPrint(
502 msg=str(f"{sku} not found in Plentymarkets export"),
503 err='', linenumber=inspect.currentframe().f_back.f_lineno)
504 return 0
505
506 return variation['Variation.id'].values.max()
507
508 def get_externalid(dataset, numberlist):
509 """
510 Parameter:
511 dataset [Ordered Dict] => data set for the item upload
512 numberlist [Dictionary] => dictionary containing path and encoding
513 of the intern number list
514 with the external id and sku
515
516 Description:
517 Open the xlsx file and retrieve the full_number field for every
518 amazon_sku field that matches to a variation within the dataset
519 """
520 if not numberlist:
521 error.warnPrint(
522 msg='No intern number list given, skip external id', err='',
523 linenumber=inspect.currentframe().f_back.f_lineno)
524 return
525
526 try:
527 extern_id = pandas.read_excel(numberlist['path'])
528 except xlrd.biffh.XLRDError as err:
529 error.errorPrint(
530 msg=str(f"..{intern['path'][-30:]} requires type [.xlsx]"),
531 err=err, linenumber=sys.exc_info()[2].tb_lineno)
532 if os.name == 'nt':
533 print("press ENTER to continue..")
534 input()
535 exit(1)
536
537 if extern_id.empty:
538 error.warnPrint(
539 msg='Internumber list is empty, skip extern_ids', err='',
540 linenumber=inspect.currentframe().f_back.f_lineno)
541
542 for key in dataset.keys():
543 if extern_id[extern_id['amazon_sku'] == key].shape[0] > 1:
544 error.warnPrint(
545 msg=str(f"multiple entries: {key} in intern numbers"),
546 err='', linenumber=inspect.currentframe().f_back.f_lineno)
547 exid = extern_id[extern_id['amazon_sku'] == key]['full_number']
548 if len(exid.index) == 0:
549 error.warnPrint(
550 msg=str(f"{key} was not found in intern number list"),
551 err='', linenumber=inspect.currentframe().f_back.f_lineno)
552 dataset[key]['ExternalID'] = exid.values.max()
File product_import.py changed (mode: 100644) (index d7299a3..e1065e0)
... ... from packages.log_files import (
23 23 from packages.gui.category_chooser import CategoryChooser from packages.gui.category_chooser import CategoryChooser
24 24 from packages.image_upload import imageUpload from packages.image_upload import imageUpload
25 25 from packages.config import assignFeatures, assignCategory, createConfig from packages.config import assignFeatures, assignCategory, createConfig
26 from packages.error import warnPrint
26 27
27 28
28 29 def main(): def main():
 
... ... def main():
37 38 sheet = {'path':'', 'encoding':''} sheet = {'path':'', 'encoding':''}
38 39 intern_number = {'path':'', 'encoding':''} intern_number = {'path':'', 'encoding':''}
39 40 stocklist = {'path':'', 'encoding':''} stocklist = {'path':'', 'encoding':''}
40 plenty_export = {'path':'', 'encoding':''}
41 plenty_export = ''
41 42 attributefile = {'path':'', 'encoding':''} attributefile = {'path':'', 'encoding':''}
43 internnumber = {'path':'', 'encoding':''}
42 44 step = 0 step = 0
43 fexc = ''
44 45
45 46 # Create a list of step names where every name fits # Create a list of step names where every name fits
46 47 # to the index of a step number # to the index of a step number
 
... ... def main():
49 50 'category-config', 'category-config',
50 51 'import-flatfile', 'import-flatfile',
51 52 'GUI', 'GUI',
52 'import-internlist',
53 53 'import-stocklist', 'import-stocklist',
54 54 'item-upload', 'item-upload',
55 55 'feature_upload', 'feature_upload',
56 56 'property_upload', 'property_upload',
57 'import-exportfile',
58 57 'image-upload' 'image-upload'
59 58 ] ]
60 59
 
... ... def main():
69 68 input_folder = config['PATH']['data_folder'] input_folder = config['PATH']['data_folder']
70 69 attribute_date = config['PATH']['file_change_date'] attribute_date = config['PATH']['file_change_date']
71 70 attributefile['path'] = config['PATH']['attribute_file'] attributefile['path'] = config['PATH']['attribute_file']
71 internnumber['path'] = config['PATH']['internnumbers']
72 plenty_export = config['PATH']['export_plentymarkets']
73
74 if not plenty_export:
75 msg = "No export URL found, enter URL @ config:'export_plentymarkets'"
76 warnPrint(msg=msg, err='',
77 linenumber=inspect.currentframe().f_back.f_lineno)
72 78
73 79 # Initial start or invalid attribute file # Initial start or invalid attribute file
74 80 if(not(attributefile['path']) or if(not(attributefile['path']) or
 
... ... def main():
84 90 with open('config.ini', 'w') as configfile: with open('config.ini', 'w') as configfile:
85 91 config.write(configfile) config.write(configfile)
86 92
93 # Initial start or invalid intern number file
94 if(not(internnumber['path']) or
95 not os.path.exists(internnumber['path'])):
96 internnumber['path'] = askopenfilename(
97 initialdir=input_folder, title="Intern number list",
98 filetypes=[ ("xlsx files", "*.xlsx") ])
99 internnumber = checkEncoding(internnumber)
100 config['PATH']['intern_number'] = internnumber['path']
101
102 with open('config.ini', 'w') as configfile:
103 config.write(configfile)
104
87 105 if not attributefile['encoding']: if not attributefile['encoding']:
88 106 attributefile = checkEncoding(attributefile) attributefile = checkEncoding(attributefile)
107 if not internnumber['encoding']:
108 internnumber = checkEncoding(internnumber)
109
89 110 features = assignFeatures(config=config) features = assignFeatures(config=config)
90 111 if not features: if not features:
91 112 exit(1) exit(1)
 
... ... def main():
156 177 elif( os.path.exists(os.path.join(upload_folder, 'log')) ): elif( os.path.exists(os.path.join(upload_folder, 'log')) ):
157 178 log_folder = os.path.join(upload_folder, 'log') log_folder = os.path.join(upload_folder, 'log')
158 179
159 step += 1
160 intern_number['path'] = askopenfilename(initialdir=input_folder,
161 title="The Intern Numbers as .csv",
162 filetypes=[ ("csv files", "*.csv") ])
163
164 intern_number = checkEncoding(intern_number)
165
166 180 step += 1; step += 1;
167 181 try: try:
168 182 stocklist['path'] = askopenfilename(initialdir=input_folder, stocklist['path'] = askopenfilename(initialdir=input_folder,
 
... ... def main():
179 193 try: try:
180 194 print("\nItem Upload\n") print("\nItem Upload\n")
181 195 itemUpload(flatfile=sheet, itemUpload(flatfile=sheet,
182 intern=intern_number,
196 intern=internnumber,
183 197 stocklist=stocklist, stocklist=stocklist,
184 198 folder=upload_folder, folder=upload_folder,
185 199 input_data=user_data, input_data=user_data,
 
... ... def main():
260 274 file_name=ntpath.basename(sheet['path'])) file_name=ntpath.basename(sheet['path']))
261 275 except OSError as err: except OSError as err:
262 276 print(err) print(err)
263 print("Missing Data, check if you have\n - a flatfile\n - a intern file table\n - export file from plentymarkets\n - a sheet with the stock numbers!\n")
264
265 # IMPORT Export FIlE
266 step += 1
267 plenty_export['path'] = askopenfilename(initialdir=input_folder,
268 title="Export File from Plentymarkets",
269 filetypes=[ ("csv files", "*.csv") ])
270
271 plenty_export = checkEncoding(plenty_export)
277 print("Missing Data, check if you have\n - a flatfile\n - a sheet with the stock numbers!\n")
272 278
273 279 step += 1 step += 1
274 280 imageUpload(flatfile=sheet, imageUpload(flatfile=sheet,
 
... ... def main():
276 282 exportfile=plenty_export, exportfile=plenty_export,
277 283 uploadfolder=upload_folder, uploadfolder=upload_folder,
278 284 filename=specific_name) filename=specific_name)
279 del fexc
280 285 # A stop in the script flow to interrupt a console window from closing itself # A stop in the script flow to interrupt a console window from closing itself
281 286 print('press ENTER to close the script...') print('press ENTER to close the script...')
282 287 input() input()
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/initBasti/Amazon2PlentySync

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/initBasti/Amazon2PlentySync

Clone this repository using git:
git clone git://git.rocketgit.com/user/initBasti/Amazon2PlentySync

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main