RocketGit

hahattpro / crowd_counting_framework (public) (License: Unspecified) (since 2020-12-23) (hash sha1)

backupfrom https://github.com/ttpro1995/crowd_counting_framework

Clone URLs: https://rocketgit.com/user/hahattpro/crowd_counting_framework ssh://rocketgit@ssh.rocketgit.com/user/hahattpro/crowd_counting_framework git://git.rocketgit.com/user/hahattpro/crowd_counting_framework

List of commits:

Subject	Hash	Author	Date (UTC)
intergrate comet ml into compact cnn	058e90a617acb76a7788e6a5d44f52563342490b	Thai Thien	2020-03-11 16:57:17
something ?!	fff52ff87af2a90452384a01bd6d6e6c4b91654e	Thai Thien	2020-03-11 15:55:04
DilatedCCNNv2	75d2989232a8a68eba9b4920ab2374ac28438e0e	Thai Thien	2020-03-10 05:11:12
fix script for ccnn_v2_t1_c2	57928056d13bc9b1f9b11e14dd305005a3a5aeea	Thai Thien	2020-03-10 04:56:33
fix trash code	33c406b13b5d45527b05dfb7f4281c3966c6471e	Thai Thien	2020-03-10 04:49:52
repair dir in config	baf522825f906a3d1fc5524f42a80da33d059640	Thai Thien	2020-03-10 04:45:11
v3 t1 c2	2d4727f47f4262833dca2087fb9e48f0d117e334	Thai Thien	2020-03-10 04:29:23
dilated ccnn v1 t1	7807d7a979353fa84d0b7319820386e93dbe5cc4	Thai Thien	2020-03-09 17:20:58
new ccnn	44a669c1f918be9d74313f29a5dbbc876c29f2fc	Thai Thien	2020-03-09 17:16:49
fix script	aa331331b12e5b454d372a550524b30a4bebe706	Thai Thien	2020-03-07 18:32:06
try reproduct ccnn with keepfull and lr 1e-5	814c520cbd1bb2d7fd50d2a8d3579d43da79fe60	Thai Thien	2020-03-07 18:30:42
my simple v4 with addition deform cnn at the end	5392aaf6c14fdd910f52096dbb921bed7470c4f7	Thai Thien	2020-03-07 18:15:22
fix the scheduler	77e6737a040f5aa5745b8a8830f5bec12322b10f	Thai Thien	2020-03-07 17:46:02
t4 lr 1e-5	acd41ed30c95f63e01a05a6d9929410637852d9e	Thai Thien	2020-03-06 19:41:49
no more lr scheduler	7289adb41de7807258eb8c29e6108fa65f59525a	Thai Thien	2020-03-06 19:35:49
reduce learning rate	bc8241e5b88b91c18bb7999a8d5d12fc79a5e3f7	Thai Thien	2020-03-06 19:28:27
dilated ccnn	5c5d92bdc0a288dd5d4ec5f1367d8cb928175bbe	Thai Thien	2020-03-06 19:04:01
done	9f05e093ec7c10284a4aedf0738f9e61d5ac6fb6	Thai Thien	2020-03-06 18:02:34
with lr scheduler	466c364b60ed22c77319b14ccc9a201614b908bf	Thai Thien	2020-03-04 17:57:49
train with learning rate scheduler	fcd5a3c8da2dd6763e0d40742edf47b49c95fcfb	Thai Thien	2020-03-04 17:55:11

Commit 058e90a617acb76a7788e6a5d44f52563342490b - intergrate comet ml into compact cnn
Author: Thai Thien
Author date (UTC): 2020-03-11 16:57
Committer name: Thai Thien
Committer date (UTC): 2020-03-11 16:57
Parent(s): fff52ff87af2a90452384a01bd6d6e6c4b91654e
Signing key:
Tree: 16ab4215f5d50cf76d70567b8c960d18b8cb2856

File	Lines added	Lines deleted
args_util.py	1	2
model_util.py	5	0
train_compact_cnn_lrscheduler.py	25	7
train_script/CCNN/ccnn_v1_t2_scheduler.sh	7	0

File args_util.py changed (mode: 100644) (index ff5520d..25625bd)
...	...	def context_aware_network_args_parse():
82	82	def my_args_parse():	def my_args_parse():
83	83	parser = argparse.ArgumentParser(description='CrowdCounting Context Aware Network')	parser = argparse.ArgumentParser(description='CrowdCounting Context Aware Network')
84	84	parser.add_argument("--task_id", action="store", default="dev")	parser.add_argument("--task_id", action="store", default="dev")
85		parser.add_argument('-a', action="store_true", default=False)
86	85
87	86	parser.add_argument('--input', action="store", type=str, default=HardCodeVariable().SHANGHAITECH_PATH_PART_A)	parser.add_argument('--input', action="store", type=str, default=HardCodeVariable().SHANGHAITECH_PATH_PART_A)
88	87	parser.add_argument('--datasetname', action="store", default="shanghaitech_keepfull")	parser.add_argument('--datasetname', action="store", default="shanghaitech_keepfull")

...	...	def my_args_parse():
90	89	# args with default value	# args with default value
91	90	parser.add_argument('--load_model', action="store", default="", type=str)	parser.add_argument('--load_model', action="store", default="", type=str)
92	91	parser.add_argument('--lr', action="store", default=1e-8, type=float)	parser.add_argument('--lr', action="store", default=1e-8, type=float)
93		parser.add_argument('--momentum', action="store", default=0.9, type=float)
	92		# parser.add_argument('--momentum', action="store", default=0.9, type=float)
94	93	parser.add_argument('--decay', action="store", default=5*1e-3, type=float)	parser.add_argument('--decay', action="store", default=5*1e-3, type=float)
95	94	parser.add_argument('--epochs', action="store", default=1, type=int)	parser.add_argument('--epochs', action="store", default=1, type=int)
96	95	parser.add_argument('--test', action="store_true", default=False)	parser.add_argument('--test', action="store_true", default=False)

File model_util.py changed (mode: 100644) (index 99475b8..6461e18)
...	...	def calculate_padding(kernel_size, dilation):
47	47	return p	return p
48	48
49	49
	50		def get_lr(optimizer):
	51		for param_group in optimizer.param_groups:
	52		return param_group['lr']
	53
	54
50	55	if __name__ == "__main__":	if __name__ == "__main__":
51	56	print(calculate_padding(kernel_size=3, dilation=4))	print(calculate_padding(kernel_size=3, dilation=4))
52	57	print(calculate_padding(kernel_size=5, dilation=1))	print(calculate_padding(kernel_size=5, dilation=1))

File train_compact_cnn_lrscheduler.py changed (mode: 100644) (index 3e3b3d0..848f4e7)
	1		from comet_ml import Experiment
	2
1	3	from args_util import my_args_parse	from args_util import my_args_parse
2	4	from data_flow import get_train_val_list, get_dataloader, create_training_image_list, create_image_list	from data_flow import get_train_val_list, get_dataloader, create_training_image_list, create_image_list
3	5	from ignite.engine import Events, create_supervised_trainer, create_supervised_evaluator	from ignite.engine import Events, create_supervised_trainer, create_supervised_evaluator

...	...	from torch import nn
12	14	from models import CompactCNN	from models import CompactCNN
13	15	import os	import os
14	16	from ignite.contrib.handlers import PiecewiseLinear	from ignite.contrib.handlers import PiecewiseLinear
	17		from model_util import get_lr
	18
	19		COMET_ML_API = "S3mM1eMq6NumMxk2QJAXASkUM"
	20		PROJECT_NAME = "crowd-counting-framework"
15	21
16	22	if __name__ == "__main__":	if __name__ == "__main__":
	23		experiment = Experiment(project_name=PROJECT_NAME)
	24
17	25	device = torch.device("cuda" if torch.cuda.is_available() else "cpu")	device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
18	26	print(device)	print(device)
19	27	args = my_args_parse()	args = my_args_parse()
	28		experiment.set_name(args.task_id)
20	29	print(args)	print(args)
	30		experiment.set_cmd_args()
	31
21	32	DATA_PATH = args.input	DATA_PATH = args.input
22	33	TRAIN_PATH = os.path.join(DATA_PATH, "train_data")	TRAIN_PATH = os.path.join(DATA_PATH, "train_data")
23	34	TEST_PATH = os.path.join(DATA_PATH, "test_data")	TEST_PATH = os.path.join(DATA_PATH, "test_data")

...	...	if __name__ == "__main__":
49	60	optimizer = torch.optim.Adam(model.parameters(), args.lr,	optimizer = torch.optim.Adam(model.parameters(), args.lr,
50	61	weight_decay=args.decay)	weight_decay=args.decay)
51	62
52		milestones_values = [(50, 1e-4), (50, 5e-5), (50, 1e-5), (50, 5e-6), (50, 1e-6), (100, 1e-7)]
	63		milestones_values = [(20, 1e-4), (30, 5e-5), (100, 1e-5), (50, 5e-6), (50, 1e-6), (100, 1e-7)]
53	64	lr_scheduler = PiecewiseLinear(optimizer, param_name="lr", milestones_values=milestones_values)	lr_scheduler = PiecewiseLinear(optimizer, param_name="lr", milestones_values=milestones_values)
54	65
55	66	trainer = create_supervised_trainer(model, optimizer, loss_fn, device=device)	trainer = create_supervised_trainer(model, optimizer, loss_fn, device=device)

...	...	if __name__ == "__main__":
57	68	metrics={	metrics={
58	69	'mae': CrowdCountingMeanAbsoluteError(),	'mae': CrowdCountingMeanAbsoluteError(),
59	70	'mse': CrowdCountingMeanSquaredError(),	'mse': CrowdCountingMeanSquaredError(),
60		'nll': Loss(loss_fn)
	71		'loss': Loss(loss_fn)
61	72	}, device=device)	}, device=device)
62	73	print(model)	print(model)
63	74

...	...	if __name__ == "__main__":
75	86	print("change lr to ", args.lr)	print("change lr to ", args.lr)
76	87	else:	else:
77	88	print("do not load, keep training")	print("do not load, keep training")
78		trainer.add_event_handler(Events.ITERATION_COMPLETED, lr_scheduler)
	89		trainer.add_event_handler(Events.EPOCH_STARTED, lr_scheduler)
79	90
80	91
81	92	@trainer.on(Events.ITERATION_COMPLETED(every=50))	@trainer.on(Events.ITERATION_COMPLETED(every=50))

...	...	if __name__ == "__main__":
90	101	metrics = evaluator.state.metrics	metrics = evaluator.state.metrics
91	102	timestamp = get_readable_time()	timestamp = get_readable_time()
92	103	print(timestamp + " Training set Results - Epoch: {} Avg mae: {:.2f} Avg mse: {:.2f} Avg loss: {:.2f}"	print(timestamp + " Training set Results - Epoch: {} Avg mae: {:.2f} Avg mse: {:.2f} Avg loss: {:.2f}"
93		.format(trainer.state.epoch, metrics['mae'], metrics['mse'], metrics['nll']))
94
	104		.format(trainer.state.epoch, metrics['mae'], metrics['mse'], metrics['loss']))
	105		experiment.set_epoch(epoch=trainer.state.epoch)
	106		experiment.log_metric("train_mae", metrics['mae'])
	107		experiment.log_metric("train_mse", metrics['mse'])
	108		experiment.log_metric("train_loss", metrics['loss'])
	109		experiment.log_metric("lr", get_lr(optimizer))
95	110
96	111	@trainer.on(Events.EPOCH_COMPLETED)	@trainer.on(Events.EPOCH_COMPLETED)
97	112	def log_validation_results(trainer):	def log_validation_results(trainer):

...	...	if __name__ == "__main__":
99	114	metrics = evaluator.state.metrics	metrics = evaluator.state.metrics
100	115	timestamp = get_readable_time()	timestamp = get_readable_time()
101	116	print(timestamp + " Validation set Results - Epoch: {} Avg mae: {:.2f} Avg mse: {:.2f} Avg loss: {:.2f}"	print(timestamp + " Validation set Results - Epoch: {} Avg mae: {:.2f} Avg mse: {:.2f} Avg loss: {:.2f}"
102		.format(trainer.state.epoch, metrics['mae'], metrics['mse'], metrics['nll']))
103
	117		.format(trainer.state.epoch, metrics['mae'], metrics['mse'], metrics['loss']))
	118		experiment.set_epoch(epoch=trainer.state.epoch)
	119		experiment.log_metric("valid_mae", metrics['mae'])
	120		experiment.log_metric("valid_mse", metrics['mse'])
	121		experiment.log_metric("valid_loss", metrics['loss'])
104	122
105	123
106	124	# docs on save and load	# docs on save and load

File train_script/CCNN/ccnn_v1_t2_scheduler.sh added (mode: 100644) (index 0000000..2d4a469)
	1	CUDA_VISIBLE_DEVICES=3 HTTPS_PROXY="http://10.30.58.36:81" nohup python train_custom_compact_cnn_lrscheduler.py \
	2	--task_id ccnn_v1_t2_scheduler \
	3	--input /data/rnd/thient/thient_data/ShanghaiTech/part_A \
	4	--lr 1e-4 \
	5	--decay 5e-5 \
	6	--datasetname shanghaitech \
	7	--epochs 300 > logs/ccnn_v1_t2_scheduler.log &

Hints:
Before first commit, do not forget to setup your git environment:

git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):

git clone https://rocketgit.com/user/hahattpro/crowd_counting_framework

Clone this repository using ssh (do not forget to upload a key first):

git clone ssh://rocketgit@ssh.rocketgit.com/user/hahattpro/crowd_counting_framework

Clone this repository using git:

git clone git://git.rocketgit.com/user/hahattpro/crowd_counting_framework

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:

... clone the repository ...
... make some changes and some commits ...
git push origin main