Evaluation Results

validation set path in dataset: /test/

test set path in dataset : /hold/

Localization Results

model	version	seed	Test Score		Validation Score
TTA			-	+	-	+
Resnet34Unet	1	0	0.6590	0.6643	0.6542	0.6590
		1	0.6690	0.6799	0.6664	0.6768
		2	0.6839	0.6903	0.6812	0.6858
		mean agg.	0.6772	--	0.6720	--
SeResnext50Unet	tuned	0	0.6963	0.7002	0.6957	0.6967
		1	0.7036	0.7074	0.6916	0.6971
		2	0.7084	0.7087	0.6981	0.7027
		mean agg.	0.7088	--	0.6998	--
Dpn92Unet	tuned	0	0.6796	0.6849	0.6776	0.6830
		1	0.6297	0.6335	0.6335	0.6322
		2	0.6708	0.6722	0.6662	0.6714
		mean agg.	0.6597	--	0.6637	--
SeNet154Unet	1	0	0.7348	0.7393	0.7261	0.7302
		1	0.7253	0.7319	0.7100	0.7163
		2	0.7326	0.7360	0.7217	0.7252
		mean agg.	0.7409	--	0.7264	--
EfficientUnetB0	Standard	0	0.7692	0.7739	0.7634	0.7666
		1	0.7685	0.7723	0.7638	0.7662
		2	0.7704	0.7740	0.7625	0.7666
	SCSE	0	0.7723	0.7749	0.7644	0.7674
		1	0.7707	0.7737	0.7628	0.7682
		2	0.7721	0.7765	0.7647	0.7711
	Wide-SE	0	0.7719	0.7758	0.7662	0.7700
	Wide-SE	1	0.7754	0.7754	0.7664	0.7682
EfficientUnetB4	Standard	0	0.7755	0.7797	0.7702	0.7724
EfficientUnetB4	SCSE	0	0.7811	0.7826	0.7718	0.7743
SegFormerB0	512*512_ade	0	0.7602	0.7281	0.7543	0.7214
		1	0.7569	0.7223	0.7533	0.7189
		2	0.7605	0.7301	0.7545	0.7250

Meta-Learning

test tasks: mexico-earthquake,joplin-tornado

model	#tasks		algorithm	meta-optimizer		inner-optimizer		shots		localization score
model	train	test	algorithm	type	lr	type	lr	support	query	localization score
EfficientUnetB0	17	2	MAML	AdamW	15e-6	SGD	1e-4	1	2	0.5372
EfficientUnetB0	17	2	MAML	AdamW	15e-6	SGD	1e-3	5	10	0.4351

Classification Results

model	version	seed	Test Score		Validation Score
TTA			-	+	-	+
Resnet34Unet	tuned	0	0.1090	0.0806	0.1119	0.0831
		1	0.1466	0.1174	0.1264	0.0997
		2	0.1314	0.1101	0.1324	0.1082
		mean agg.	0.0860	--	0.0832	--
SeResnext50Unet	tuned	0	0.6164	0.6152	0.6397	0.6347
		1	0.6135	0.6069	0.6012	0.5991
		2	0.6319	0.6422	0.6271	0.6361
		mean agg.	0.6360	--	0.6301	--
Dpn92Unet	tuned	0	0.6564	0.6657	0.6387	0.6441
		1	0.6233	0.6343	0.5869	0.5813
		2	0.6246	0.6252	0.6075	0.6138
		mean agg.	0.6460	--	0.6258	--
SeNet154Unet	tuned	0	0.6916	0.7034	0.6684	0.6722
		1	0.6216	0.6342	0.5889	0.6123
		2	0.6868	0.6949	0.6520	0.6479
		mean agg.	0.6954	--	0.6596	--
EfficientNetB0	Standard	0	0.7576	0.7571	0.7606	0.7505
	SCSE	0	0.7591	0.7525	0.7497	0.7399
	Wide-SE	0	0.7726	0.7667	0.7769	0.7737
EfficientUnetB4	Standard	0	0.7732	0.7679	0.7684	0.7589
EfficientUnetB4	SCSE	0	0.7746	0.7650	0.7740	0.7635
SegFormer	Standard	0	0.7574	0.7380	0.7385	0.6993