hahattpro / crowd_counting_framework (public) (License: Unspecified) (since 2020-12-23) (hash sha1)
backupfrom https://github.com/ttpro1995/crowd_counting_framework
Clone URLs: https://rocketgit.com/user/hahattpro/crowd_counting_framework ssh://rocketgit@ssh.rocketgit.com/user/hahattpro/crowd_counting_framework git://git.rocketgit.com/user/hahattpro/crowd_counting_framework
debug/a-1 develop feature/add_can_adcrowdnet feature/add_context_aware_network feature/attn_can_adcrowd_simple feature/cache_dataloader feature/compact_cnn feature/compare_with_original_count feature/dataloader_not_divide_8 feature/debug_viz feature/documentation feature/improve_density_gen feature/improve_loss_fn feature/loader_full_and_augment feature/refactor-dataflow feature/refactor_ccnn_basecode feature/save_best feature/simple_cometml feature/train_context_aware_network_on_shtA feature/train_dev_test_split hotfix/fix_visualize_data_loader master meow_exp server-training with_apex
List of commits:
Subject Hash Author Date (UTC)
add data path 80134de767d0137a663f343e4606bafc57a1bc1f Thai Thien 2020-02-02 03:38:21
test if ShanghaiTech datapath is correct 97ee84944a4393ec3732879b24f614826f8e7798 Thai Thien 2020-02-01 03:57:31
refactor and test ShanghaiTech datapath 9542ebc00f257edc38690180b7a4353794be4019 Thai Thien 2020-02-01 03:53:49
fix the unzip flow b53c5989935335377eb6a88c942713d3eccc5df7 Thai Thien 2020-02-01 03:53:13
data_script run seem ok 67420c08fc1c10a66404d3698994865726a106cd Thai Thien 2020-02-01 03:33:18
add perspective 642d6fff8c9f31e510fda85a7fb631fb855d8a6d Thai Thien 2019-10-06 16:54:44
fix padding with p 86c2fa07822d956a34b3b37e14da485a4249f01b Thai Thien 2019-10-06 02:52:58
pacnn perspective loss fb673e38a5f24ae9004fe2b7b93c88991e0c2304 Thai Thien 2019-10-06 01:38:28
data_flow shanghaitech_pacnn_with_perspective seem working 91d350a06f358e03223966297d124daee94123d0 Thai Thien 2019-10-06 01:31:11
multiscale loss and final loss only mode c65dd0e74ad28503821e5c8651a3b47b4a0c7c64 Thai Thien 2019-10-05 15:58:19
wip : perspective map eac63f2671dc5b064753acc4f40bf0f9f216ad2a Thai Thien 2019-10-04 16:26:56
shell script f2106e700b6f6174d4dd276f25ec6f3d9ff239bb thient 2019-10-04 07:42:51
WIP 42c7c8e1d772fbbda61a4bdf9e329f74e1efb600 tthien 2019-10-03 17:52:47
add readme 580cf43d1edddd67b1f6a2c57fdd5cee3dba925c Thai Thien 2019-10-02 17:44:49
update script, debug ddb68b95389be1c1d398118677dd227a8bb2b70b Thai Thien 2019-10-02 15:52:31
add d (output density map) to loss function) a0c71bf4bf2ab7393d60b06a84db8dfbbfb1a6c2 tthien 2019-09-30 16:32:39
fix the args, add save interval for model, so we don't save them all 9fdf9daa2ac4bd12b7b62521d81e520db0debd01 tthien 2019-09-30 16:30:00
meow 1ad19a22a310992e27a26471feeb37375124d075 tthien 2019-09-29 18:25:43
fix pacnn perspective map 453ece3ccb818889ba895bfc4285f7905d33cba5 Thai Thien 2019-09-25 17:20:33
apex not work so well da8c0dd57297f972201f31d57e66897177922f48 Thai Thien 2019-09-24 17:25:59
Commit 80134de767d0137a663f343e4606bafc57a1bc1f - add data path
Author: Thai Thien
Author date (UTC): 2020-02-02 03:38
Committer name: Thai Thien
Committer date (UTC): 2020-02-02 03:38
Parent(s): 97ee84944a4393ec3732879b24f614826f8e7798
Signing key:
Tree: eb5f705d1d0977a9cf0bf940f9170508949a0b4d
File Lines added Lines deleted
data_util/__init__.py 0 0
data_util/data_path.py 68 0
data_util/test_data_path.py 81 0
data_util/unittest_data/mock_shanghaitech/part_A/test_data/ground-truth-h5/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_A/test_data/ground-truth/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_A/test_data/images/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_A/test_data/pmap/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_A/train_data/ground-truth-h5/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_A/train_data/ground-truth/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_A/train_data/images/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_A/train_data/pmap/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_B/test_data/ground-truth-h5/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_B/test_data/ground-truth/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_B/test_data/images/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_B/test_data/pmap/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_B/train_data/ground-truth-h5/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_B/train_data/ground-truth/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_B/train_data/images/place_holder 2 0
data_util/unittest_data/mock_shanghaitech/part_B/train_data/pmap/place_holder 2 0
hard_code_variable.py 1 1
File data_util/__init__.py copied from file playground/__init__.py (similarity 100%)
File data_util/data_path.py added (mode: 100644) (index 0000000..451beb1)
1 """
2 Store data path here
3 """
4 import os
5
6 SHANGHAITECH_PATH = "/home/tt/project/crowd-counting-data/ShanghaiTech"
7
8
9 class ShanghaiTechDataPath:
10
11 def __init__(self, root=SHANGHAITECH_PATH):
12 self.root = root
13 self.part_a_path = os.path.join(self.root, "part_A")
14 self.part_b_path = os.path.join(self.root, "part_B")
15
16 def get(self):
17 return self.root
18
19 def get_a(self):
20 return _ShanghaiTechPart(self.part_a_path)
21
22 def get_b(self):
23 return _ShanghaiTechPart(self.part_b_path)
24
25
26 class _ShanghaiTechPart:
27 def __init__(self, root):
28 self.root = root
29 self.train_path = os.path.join(self.root, "train_data")
30 self.test_path = os.path.join(self.root, "test_data")
31
32 def get_train(self):
33 return _ShanghaiTechData(self.train_path)
34
35 def get_test(self):
36 return _ShanghaiTechData(self.test_path)
37
38
39 class _ShanghaiTechData:
40
41 def __init__(self, root):
42 self.root = root
43 self.images_path = os.path.join(self.root, "images")
44 self.ground_truth_mat_path = os.path.join(self.root, "ground-truth")
45 self.ground_truth_h5_path = os.path.join(self.root, 'ground-truth-h5')
46 self.perspective_path = os.path.join(self.root, "pmap")
47
48 def get(self):
49 return self.root
50
51 def get_images(self):
52 return self.images_path
53
54 def get_anotation_point(self):
55 return self.ground_truth_mat_path
56
57 def get_density_map(self):
58 return self.ground_truth_h5_path
59
60 def get_perspective_map(self):
61 return self.perspective_path
62
63
64 # SHANGHAITECH_PART_A = os.path.join(SHANGHAITECH, "part_A")
65 # SHANGHAITECH_PART_A_train = os.path.join(SHANGHAITECH_PART_A, "part_A")
66 #
67 #
68 # SHANGHAITECH_PART_B = os.path.join(SHANGHAITECH, "part_B")
File data_util/test_data_path.py added (mode: 100644) (index 0000000..115adc0)
1 from unittest import TestCase
2 import os
3
4 from hard_code_variable import HardCodeVariable
5 from .data_path import ShanghaiTechDataPath
6 import glob
7
8
9 class TestShanghaiTechDataPath(TestCase):
10
11 def test_get(self):
12 self.test_root = "unittest_data/mock_shanghaitech"
13 self.data = ShanghaiTechDataPath(root=self.test_root)
14 print(self.data.get())
15
16 def test_get_a(self):
17 self.test_root = "unittest_data/mock_shanghaitech"
18 self.data = ShanghaiTechDataPath(root=self.test_root)
19 print(self.data.get_a())
20
21 def test_get_b(self):
22 self.test_root = "unittest_data/mock_shanghaitech"
23 self.data = ShanghaiTechDataPath(root=self.test_root)
24 print(self.data.get_b())
25
26 def test_get_hc(self):
27 hc = HardCodeVariable()
28 self.test_root = hc.SHANGHAITECH_PATH
29 self.data = ShanghaiTechDataPath(root=self.test_root)
30 print(self.data.get())
31
32 def test_get_a_hc(self):
33 hc = HardCodeVariable()
34 self.test_root = hc.SHANGHAITECH_PATH
35 self.data = ShanghaiTechDataPath(root=self.test_root)
36 print(self.data.get_a())
37
38 def test_get_b_hc(self):
39 hc = HardCodeVariable()
40 self.test_root = hc.SHANGHAITECH_PATH
41 self.data = ShanghaiTechDataPath(root=self.test_root)
42 print(self.data.get_b())
43
44 def testIntergration(self):
45 hard_code = HardCodeVariable()
46 self.test_root = hard_code.SHANGHAITECH_PATH
47 self.data = ShanghaiTechDataPath(root=self.test_root)
48 if os.path.exists(self.data.get()):
49 print("exist " + self.data.get())
50 print("let see if we have train, test folder")
51 train_path_a = self.data.get_a().get_train().get()
52 train_path_b = self.data.get_b().get_train().get()
53 test_path_a = self.data.get_a().get_test().get()
54 test_path_b = self.data.get_a().get_test().get()
55 if os.path.exists(train_path_a):
56 print("exist " + train_path_a)
57 if os.path.exists(train_path_b):
58 print("exist " + train_path_b)
59 if os.path.exists(test_path_a):
60 print("exist " + test_path_a)
61 if os.path.exists(test_path_b):
62 print("exist " + test_path_b)
63 print("count number of image")
64 image_folder_list = [train_path_a, train_path_b, test_path_a, test_path_b]
65 for image_root_path in image_folder_list:
66 image_path_list = glob.glob(os.path.join(image_root_path, "images", "*.jpg"))
67 density_path_list = glob.glob(os.path.join(image_root_path, "ground-truth-h5", "*.h5"))
68 count_img = len(image_path_list)
69 count_density_map = len(density_path_list)
70 first_img = image_path_list[0]
71 first_density_map = density_path_list[0]
72 print("in folder " + image_root_path)
73 print("--- total image" + str(count_img))
74 print('--- first img ' + first_img)
75 print("--- total density map " + str(count_density_map))
76 print("--- first density map " + str(first_density_map))
77 if count_img == count_density_map:
78 print("--- number of density map = number of image")
79 else:
80 print("xxxxx number of density map !!!!!= number of image")
81 assert count_img == count_density_map
File data_util/unittest_data/mock_shanghaitech/part_A/test_data/ground-truth-h5/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_A/test_data/ground-truth/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_A/test_data/images/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_A/test_data/pmap/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_A/train_data/ground-truth-h5/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_A/train_data/ground-truth/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_A/train_data/images/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_A/train_data/pmap/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_B/test_data/ground-truth-h5/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_B/test_data/ground-truth/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_B/test_data/images/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_B/test_data/pmap/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_B/train_data/ground-truth-h5/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_B/train_data/ground-truth/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_B/train_data/images/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File data_util/unittest_data/mock_shanghaitech/part_B/train_data/pmap/place_holder added (mode: 100644) (index 0000000..80816f6)
1 This piece of text make the file exist in git
2
File hard_code_variable.py changed (mode: 100644) (index 940fe25..b09b84e)
... ... if __name__ == "__main__":
20 20 """ """
21 21 Test if path is working Test if path is working
22 22 """ """
23 hard_code =HardCodeVariable()
23 hard_code = HardCodeVariable()
24 24 if os.path.exists(hard_code.SHANGHAITECH_PATH): if os.path.exists(hard_code.SHANGHAITECH_PATH):
25 25 print("exist " + hard_code.SHANGHAITECH_PATH) print("exist " + hard_code.SHANGHAITECH_PATH)
26 26 print("let see if we have train, test folder") print("let see if we have train, test folder")
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/hahattpro/crowd_counting_framework

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/hahattpro/crowd_counting_framework

Clone this repository using git:
git clone git://git.rocketgit.com/user/hahattpro/crowd_counting_framework

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main