beir/scidocs

목차

1. 사용법

1.1. 모든 데이터 순회

1.2. 개별 데이터 접근

2. 속성

2.1. doc

2.2. query

2.3. qrel

3. 통계

4. 인용

5. 출처

6. 라이센스



1. 사용법

1.1. 모든 데이터 순회

from hamu_tool.dataset import DataLoader

loader = DataLoader.load('beir/scidocs')

for doc in loader.get_docs():
    print(doc.id, doc.text, doc.title, doc.authors, doc.year, doc.cited_by, doc.references)
    break

for query in loader.get_queries():
    print(query.id, query.text, query.authors, query.year, query.cited_by, query.references)
    break

for qrel in loader.get_qrels('[mode]'):
    print(qrel.qid, qrel.did, qrel.score)
    break

1.2. 개별 데이터 접근

from hamu_tool.dataset import DataLoader

loader = DataLoader.load('beir/scidocs')

doc = loader.get_doc('[did]')
print(doc)

query = loader.get_query('[qid]')
print(query)

qrel = loader.get_qrel('[mode]', '[qid]')
print(qrel)

2. 속성

2.1. doc

속성자료형
idstr
textstr
titlestr
authorslist[str]
yearint
cited_bylist[str]
referenceslist[str]

2.2. query

속성자료형
idstr
textstr
authorslist[str]
yearint
cited_bylist[str]
referenceslist[str]

2.3. qrel

속성자료형
qidstr
didstr
scoreint
  • [mode]: test

3. 통계

수치
TaskCitation Prediction
DomainScientific
# Query1,000
# Doc25,313
# Qreltest4,741
Average Rel D/Qtest4.74
Average Query Length (words)9.44
Average Doc Length (words)169.51

4. 인용

@inproceedings{Cohan2020Scidocs,
    title = "{SPECTER}: Document-level Representation Learning using Citation-informed Transformers",
    author = "Cohan, Arman and Feldman, Sergey and Beltagy, Iz  and Downey, Doug  and Weld, Daniel",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.acl-main.207",
    doi = "10.18653/v1/2020.acl-main.207",
    pages = "2270--2282"
}
@article{Thakur2021Beir,
  title = "BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models",
  author = "Thakur, Nandan and Reimers, Nils and Rücklé, Andreas and Srivastava, Abhishek and Gurevych, Iryna", 
  journal = "arXiv preprint arXiv:2104.08663",
  month = "4",
  year = "2021",
  url = "https://arxiv.org/abs/2104.08663",
}

5. 출처


6. 라이센스