박부덕 페르소나 기준
연구실 관련 15+
수업 조교 관련 10+
교내 연락 15+
광고, 홍보, 안내 10+
50개
화 13, 수 15, 목 12, 금 14
각 지표별 설명
ROUGE-1 | BERTScore | G-EVAL v1 | G-EVAL v2 | G-EVAL v3 | |
---|---|---|---|---|---|
v1 | P=0.0667, R=0.0042, F1=0.1678 | P=0.8223, R=0.8789, | |||
F1=0.8494 | cn=4.3333, ch=4.6875, rel=4.5833, cc=4.3958 | ||||
v1 + refine | P=0.2170, R=0.2639, F1=0.5516 | P=0.8346, R=0.9012, F1=0.8664 | cn=4.3125, ch=4.7083, rel=4.7083, cc=4.6875 | ||
v2 | P=0.0725, R=0.0042, F1=0.1518 | P=0.8391, R=0.8879, | |||
F1=0.8624 | cn=4.4792, ch=4.6875, rel=4.6875, cc=4.7500 | ||||
v2 + one shot | P=0.0744, R=0.0000, F1=0.1191 | P=0.8547, R=0.8938,F1=0.8735 | cn=4.4792, ch=4.3333, rel=4.5833, cc=4.8542 | ||
v2 + refine | P=0.2839, R=0.2865, F1=0.5417 | P=0.8515, R=0.9133, F1=0.8806 | cn=4.7083, ch=4.8542, rel=4.7917, cc=4.8125 | ||
v2 + refine + one shot | P=0.3181, R=0.2569, F1=0.4953 | P=0.8818, R=0.9296,F1=0.9046 | cn=4.5000, ch=4.2708, rel=4.5625, cc=4.8750 | ||
v3 | P=0.2011, R=0.1362, F1=0.3567 | P=0.8273, R=0.8849,F1=0.8539 | cn=4.3542, ch=3.3125, rel=4.3542, cc=4.8542 | ||
v3 + one shot | P=0.2288, R=0.2005, F1=0.3661 | P=0.8325, R=0.8905,F1=0.8588 | cn=4.2292, ch=3.0417, rel=4.1250, cc=4.9375 | ||
v3 + refine | P=0.2618, R=0.2049, F1=0.4649 | P=0.8740, R=0.9146,F1=0.8932 | cn=4.3333, ch=3.2708, rel=4.2917, cc=4.8750 | ||
v3 + refine + one shot | P=0.3062, R=0.2691, F1=0.4690 | P=0.8905, R=0.9319, F1=0.9101 | cn=4.2083, ch=3.2083, rel=4.1667, cc=4.9167 |
ROUGE-1 | BERT Score | G-EVAL | |||||
---|---|---|---|---|---|---|---|
Recall | Precision | F1 | Recall | Precision | F1 | Conciseness | |
prompt1 | 0.0667 | 0.0042 | 0.1678 | 0.8223 | 0.8789 | 0.8494 | 4.3958 |
prompt1, refine | 0.2170 | 0.2639 | 0.5516 | 0.8346 | 0.9012 | 0.8664 | 4.6875 |
prompt2 | 0.0725 | 0.0042 | 0.1518 | 0.8391 | 0.8879 | 0.8624 | 4.7500 |
prompt2, refine | 0.2839 | 0.2865 | 0.5417 | 0.8515 | 0.9133 | 0.8806 | 4.8125 |
prompt2, one-shot | 0.0744 | 0.0000 | 0.1191 | 0.8547 | 0.8938 | 0.8735 | 4.8542 |
prompt2, refine, one-shot | 0.3181 | 0.2569 | 0.4953 | 0.8818 | 0.9296 | 0.9046 | 4.8750 |
prompt3 | 0.2011 | 0.1362 | 0.3567 | 0.8273 | 0.8849 | 0.8539 | 4.8542 |
prompt3, refine | 0.2618 | 0.2049 | 0.4649 | 0.8740 | 0.9146 | 0.8932 | 4.8750 |
prompt3, one-shot | 0.2288 | 0.2005 | 0.3661 | 0.8325 | 0.8905 | 0.8588 | 4.9375 |
prompt3, refine, one-shot | 0.3062 | 0.2691 | 0.4690 | 0.8905 | 0.9319 | 0.9101 | 4.9167 |
ROUGE 값 상승